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Introduction 


by Francis T. Hoban, William M. Lawbaugh and Edward J. Hoffman 


In a classic fable on individual perspective, 
several blind men are touching different 
parts of an elephant. Each individual is cer- 
tain of something different as the object or 
animal is identified. All of them, of course, 
are wrong and in complete disagreement as 
to what they are handling. The moral of the 
story is the importance of always getting a 
variety of perspectives before coming to a de- 
cision, and realizing that none of them may 
be fully accurate or complete. 

When it comes to the management of com- 
plex projects, it seems that such a lesson can 
never be forgotten. In particular, when it 
comes to the issue of program control, there 
are so many different possibilities, perspec- 
tives and opinions that it is easy to confuse 
the parts for the whole. 

In the present environment of increasing 
scrutiny and budget reductions, it is critical 
that this area be integrated effectively in the 
management of any project. This collection of 
readings is aimed at shedding some light and 
offering a diversity of views in this area. 

In contrast to the term ‘systems engineering,’ 
there has been general agreement in NASA 
as to what functions should be included un- 
der the term ‘program control’ and a lively 
debate about where program control func- 
tions should reside and who they should re- 
port to. Despite this ongoing debate, projects 
continue to be managed successfully; pro- 
gram control functions are being performed 
at NASA and status is being communicated 
to project management teams. 

In NASA’s earlier days, particularly during 
the Apollo era, the performance and schedule 
aspects of program control were developed to 
a fine art, with cost of far less importance. 


Performance and schedule functions were ab- 
solutely vital to the success of the Apollo pro- 
gram. NASA led the world in its ability to 
manage a large, complex system using the 
relatively new and innovative techniques of 
program control. 

Post-Apollo NASA has applied program con- 
trol processes in a less rigorous manner, 
largely because the many smaller projects 
that followed could not afford the enormous 
costs associated with NASA’s complex con- 
trol systems. With the initiation of the Low 
Cost Systems effort in the mid-1970s, cost es- 
timation and control became paramount. 
Cost has been a full partner to schedule and 
performance functions ever since. 

Readings in Program Control, like its com- 
panion volume, Readings in Systems Engi- 
neering (SP-6102, 1993), is primarily for the 
next generation of project managers. As part 
of the corporate memory of the NASA Pro- 
gram and Project Management Initiative, 
this volume attempts to collect, preserve and 
transmit the lessons learned on program con- 
trol for better management of the future. It is 
a well known fact of human nature that 
those who fail to learn from the successes and 
failures of the past are destined to repeat 
past mistakes. 

We begin with the former Comptroller of 
NASA, Bill Lilly, who “wrote the book” on 
Program Control for the Agency. Kelley Cyr 
of the Johnson Space Center follows with the 
latest on cost estimating methods. 

These overview pieces are followed by a 
group of articles on planning and scheduling. 
David Christensen, a professor of accounting 
at the Air Force Institute of Technology, ex- 
amines cost overruns in the defense industry. 
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Dorothy Pennington and Walt Majerowicz 
look at program control in a current project. 
Pennington joined the Tropical Rainfall Mis- 
sion as Deputy Project Manager, Resources; 
joint author Majerowicz of Computer Sci- 
ences Corporation is Schedule Manager. 

Nancy Abell, Chief of Administration and 
Resource Management at Goddard Space 
Flight Center’s Space Sciences Directorate, 
describes Goddard’s Performance Measure- 
ment System. Gerry Longanecker, Senior 
Principal Staff Member in the Space Science 
and Applications Group of BDM Internation- 
al, Inc., explains scoping a program and 
scheduling. He is former Director of Flight 
Projects at GSFC. 

A landmark piece by former NASA Deputy 
Administrator George M. Low closes out this 
book of readings. This piece is based upon a 
speech he gave in 1972 to a joint symposium 
in Washington of the Armed Forces Manage- 
ment Association and the National Security 
Industrial Association. As a consequence of 
these innovative ideas, Low set up NASA’s 
short-lived, useful Low Cost Systems Office. 

Two of the readings are excerpted from clas- 
sic texts in the evolution of program control. 
The 1968 scheduling guide for program man- 
agers from the Defense Systems Manage- 
ment College contains excellent but hard to 
find charts and graphs for better program 
control. Marshall Space Flight Center’s ap- 
proach to program control development is 
one of the best available anywhere. 

John D. Hodge, respected author, and veter- 
an cost estimator Joe Hamaker of Marshall 
Space Flight Center, survey the history of es- 
timating through various cultural changes. 
Humboldt Mandell, Jr., Head of the Explora- 
tion Programs Office at Johnson Space Cen- 


ter, defines the major issues of program man- 
agement related to cost estimation practices. 
Bill Keathley, former Associate Director of 
Programs at GSFC, offers nine advantages of 
cost plus award fee contracts. Ignacio Man- 
zanera, CCE, with ARAMCO in Saudia Ara- 
bia, describes the planning and scheduling 
systems with easy-to-follow tables. 

John R. Biggs and Walter J. Downhower 
show how program control techniques led to 
the notable success of a NASA development 
project, the 1973 Mariner Venus/Mercury 
Project. Downhower was the Systems Design 
and Integration Section Chief at the Jet Pro- 
pulsion Laboratory. John Biggs had worked 
in procurement for the U.S. Air Force and 
later for NASA’s Office of Space Science. 

The editors gratefully acknowledge the au- 
thors for sharing their information with us 
and with you. We also wish to thank the Pro- 
gram/Project Management Initiative (PPMI) 
Librarian, Jeffrey Michaels, for his support. 
Thanks must also go out to the entire NASA 
family for their contributions to the art and 
science of program control. In this age of new 
program constraints and challenges, we 
agree with George Low, who thought that ef- 
fective program control “is as great a techno- 
logical challenge as everything else we have 
done in space.” 

Finally, the editors thank George Mason 
University, where two of us served as Senior 
Fellows in The Institute of Public Policy 
(TIPP) in 1993 and 1994 during much of the 
research and editing of this volume. Thank 
you, especially, to TIPP leaders Kingsley 
Haines and Roger Stough for the accommo- 
dations at the Center for Innovative Technol- 
ogy in Herndon, Virginia, a grand place to 
think and work. The views of the authors are 
their own. 
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Program Control in NASA: Needs and Opportunities 

by a Study Team of the National Academy of Public Administration 
William E. Lilly, Project Director 


The National Aeronautical and Space Ad- 
ministration (NASA) has successfully man- 
aged some of this country’s most complex 
technology and development programs. 
These successes have included the applica- 
tion of sound program control processes. The 
impetus for this study arose from the NASA 
Management Study Group findings that over 
time, some program control tools and disci- 
plined procedures and processes had weak- 
ened. The Study Group recommended that 
steps be taken to establish a comprehensive 
training approach in program management, 
and specifically, in program control func- 
tions. This study looks at program control 
processes within NASA currently in use, de- 
fines a “model” of program control functions, 
and provides recommendations on program 
control training needs and opportunities. 

In 1988, NASA Headquarters tasked the Na- 
tional Academy of Public Administration 
(NAPA) to examine the processes and sys- 
tems used by NASA to manage and control 
program and project activities. Essential ele- 
ments of a program control system include 
program development planning and docu- 
menting program requirements; integrated 
scheduling; resources management; configu- 
ration management; documentation and 
data management; establishment of essen- 
tial baselines; and the conduct of perfor- 
mance reviews. Specifically the NAPA study 
was designed to include: 

• Determination and definition of program 
control functions as currently practiced in 
NASA. 

• Definition of a model of program control 
functions for NASA. 

• Observations on training of personnel. 


• Generation of recommendations for train- 
ing in program control objectives and pro- 
cesses at the basic, intermediate and ad- 
vanced levels of project management. 

The impetus for a program control aspect of 
program and project management training 
and developmental efforts can be traced to a 
series of findings and recommendations on 
strengthening program management and 
control functions which were derived from 
the Rogers Commission and the NASA Man- 
agement Study Group (the Phillips Commit- 
tee) reports. In reviewing the total function 
of NASA program management, the Phillips 
Committee found the weakest area to be that 
of program planning and control. Committee 
members commented that over time NASA’s 
use of program control tools and disciplined 
procedures and processes had weakened. 
They recommended the reinstitution of a 
Program Approval Document system and a 
revitalized hierarchy of program/project sta- 
tus reviews against approved baselines. In 
addition, the Study Group recommended that 
steps be taken to develop a comprehensive 
training approach in program management, 
specifically in program control functions, 
that would be based on real experience. 

The significance of the program control func- 
tions within NASA cannot be overstated. 
The success of large and complex research 
and development projects depends on com- 
mitment, diligent and disciplined attention 
to numerous planning, resource and schedul- 
ing variables, and the integration and bal- 
ancing of complex, interrelated activities. 
Along with the systems engineering func- 
tion, the program control function is one of 
the most important activities in successful 
program/project management. Systematic 
and disciplined attention to the implications 


1 


READINGS IN PROGRAM CONTROL 


of variances between planned baselines and 
actual performance on development projects 
is critical to taking early remedial action, re- 
ducing costly delays and achieving success. 

The purpose of this study is to indicate the 
areas of need as well as provide guidance to 
the development of training opportunities 
concerned with program control in support of 
effective program/project management in 
NASA. The study could not have been com- 
pleted without the assistance of NASA em- 
ployees at field Centers and Headquarters. 
Their contributions helped the staff to under- 
stand the application of project control func- 
tions at different Centers. Special thanks are 
owed to Frank Hoban, Program Manager, 
NASA Program and Project Management 
Initiative, who provided the Academy with 
the environment to pursue the study. 

The Program Control Function 

In NASA, a project “. . . is a defined, time- 
limited activity with clearly established ob- 
jectives and boundary conditions executed to 
gain knowledge, create a capability, or pro- 
vide a service.” Major space research and de- 
velopment projects in NASA typically in- 
clude design, development, fabrication, test, 
and flight operations. A program/project 
manager is designated responsible for ensur- 
ing the performance of all functions neces- 
sary for management of the project. The 
three basic elements of the manager’s job are 
technical performance, cost and schedule. 
The program/project manager needs to know 
where the project is at any point in time and 
to identify and scope problems early. Pro- 
gram/project control, which aids the project 
manager in this regard, is the total manage- 
ment process of establishing and maintain- 
ing program baselines and effectively sup- 
porting the project manager in meeting the 
overall objectives of the project. 

The combination of functions of program con- 
trol is an essential element of the program 
management process. The establishment of 


comprehensive performance requirements by 
systems engineering provides the details and 
parameters necessary for program control to 
maintain a comprehensive, adequately ex- 
plicit and integrated program plan. This plan 
documents and defines program require- 
ments and establishes the official baselines 
of program content, scope, configuration, 
schedule and cost. A comprehensive program 
control process includes procedures for re- 
porting and reviewing performance against 
baselines; analyzing and synthesizing pro- 
gram performance; evaluating alternatives; 
developing disciplined processes for consider- 
ing, approving, and implementing changes to 
official baselines; and assuring positive feed- 
back on all directions and decisions. It also 
provides a uniform system of program docu- 
mentation and assures clear and consistent 
communications throughout the program 
community on program progress, status, and 
issues. The integrated operation of these 
functions furnishes the means to determine 
the harmony of actual and planned cost, 
schedule, and performance goals during de- 
velopment and fabrication by verifying 
whether everything is occurring in accord 
with baseline plans. The larger point is clear: 
a program control system requires sustained 
attention to the system as a totality, rather 
than as a group of parts. 

Ultimate responsibility for the effectiveness 
of program management control rests with 
top management. Top management decides 
upon Agency strategy, policy, and organiza- 
tional and accountability structure. The con- 
trol system is a set of major tools and proce- 
dures for implementing those decisions and 
for forming coherent and defensible strate- 
gies to cope with changed and changing cir- 
cumstances. For the most effective program 
management and control to exist, an envi- 
ronment of accountability of organizations 
and individuals needs to exist at the top of 
the Agency. It should be clear to the entire 
Agency how NASA intends to operate and 
what is expected of all elements. Delegations 
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of authority, definitions of roles and assign- 
ments of responsibilities should carry with 
them the terms of accountability. Disciplined 
processes for obtaining required feedback on 
delegations and for measuring and system- 
atically reviewing performance on programs 
and projects should exist. The pattern of pro- 
gram reviews against approved program 
baselines should also be established at the 
top. This can consist of separate reviews or be 
a part of the general management review 
process, but a disciplined approach of review- 
ing status against approved baselines by the 
Administrator and/or the Deputy is needed. 
The strength of such an approach is that it 
allows Agency leaders to directly, program- 
matically and effectively keep tabs on the 
performance and potential pitfalls of pro- 
grams. This in turn enables top managers to 
identify and consider the implications of both 
“inside” factors and “outside” factors, forces 
and trends which are likely to have an effect 
on NASA and its missions. 

A number of characteristics distinguish 
NASA research and development projects, 
including: 

• Uncertainty. Many of the processes and 
products to be developed will be under- 
taken for the first time and all compo- 
nents require the performance of ad- 
vanced technologies. 

• Long lead times in development and fabri- 
cation. This necessitates concurrent de- 
velopment of elements and subsystems 
and the fitting of end products together. It 
requires a high order of advance planning 
and detailed monitoring and tracking and 
increases the need for testing (component 
testing, subsystem testing and system 
testing). 

• Size and complexity of projects and the 
large number and dispersion of partici- 
pants. 


• Persistent scrutiny of projects by the pub- 
lic, the Congress and the scientific com- 
munity. Not only must the work be done 
well, the project manager must be pre- 
pared to interpret, explain and defend 
what is being done and why. Practices 
and standards for public projects far ex- 
ceed typical industry standards. 

Against this background, it is important to 
keep in mind that good program manage- 
ment is a matter of balancing different inter- 
nal and external factors so that performance 
is maximized over the longer term. Program 
control interventions, if used correctly, help 
to maintain this balance. 

Major Functions of Program/Project 
Control 

The basic control functions for development 
projects are planning, configuration manage- 
ment, scheduling, resource management and 
data management. In some cases procure- 
ment activities and other business manage- 
ment activities may become part of the con- 
trol function, as well as logistics and sepa- 
rate activities for program analysis, manage- 
ment information and program reviews. The 
combination of activities included depends 
upon the size and complexity of the program 
or project, the existing support structure and 
the preferences of the Centers and the indi- 
vidual managers. Regardless of the individ- 
ual functions, more than anything else in 
program control, it is important for the per- 
sonnel to see and comprehend the totality of 
the job to be done and to thoroughly under- 
stand the interrelationships and interfaces of 
the subsystems and systems, as well as the 
organizations and participants in the project. 
Another important element in structuring 
and carrying out program control functions 
is uncertainty and the inability to completely 
eliminate it. Uncertainty should be specifi- 
cally considered in program planning, sched- 
uling and resource planning. 
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Program Plans and Requirements 
The development plan is the basic plan for 
execution of the program or project. It is the 
top-level requirements document and the 
top-level implementation plan. It is the sin- 
gle authoritative summary document that 
sets forth the manner in which the objectives 
shall be accomplished. It defines the program 
organization, responsibilities, requirements, 
resources and time phasing of the major ac- 
tions required. 

Program planning sets forth the develop- 
ment requirements needed to establish and 
maintain an integrated planning baseline of 
what is to be done, how it is to be done and 
when it is to be done. It is not a one-time pro- 
cess, since the development of detailed per- 
formance requirements are not established 
at one point in time. In addition to the tech- 
nical requirements, detailed management 
and mission requirements should be estab- 
lished. It is a continuing process of laying out 
and ensuring a unified effort in implement- 
ing the program, adjusting to changing con- 
ditions, maintaining the program or project 
development plan, and integrating ongoing 
technical requirements. Although planning 
steps are laid out in a linear sequential man- 
ner, the process is iterative. 

The technical requirements establish the 
work packages. The development of the pro- 
ject work breakdown structure (WBS), con- 
sistent with the Agency coding structure, 
may also occur in conjunction with the plan- 
ning function or it may be part of one of the 
other functions. On NASA development pro- 
jects the WBS will normally be end-item ori- 
ented rather than discipline oriented. 

Resources Management 
Resources management includes the estab- 
lishment, monitoring, and maintenance of 
obligation and cost as well as the manpower 
baselines. Manpower constitutes the vast 
majority of development costs, and knowl- 
edge of status and trends are extremely im- 


portant. The reporting structure for cost 
should be established and maintained with 
an emphasis on cost phasing and cost to com- 
pletion. Reporting systems and selection of 
report items should be designed to raise 
questions, not to answer them; the implica- 
tions are important, not the absolute value 
recorded. The absolute value is useful , only 
for historical and legal purposes. 

The planning of reportable items is usually 
achieved through use of the Work Break- 
down Structure accounts. The structure and 
analysis of report implications should be cor- 
related closely with schedule and technical 
performance. The recording and reporting of 
cost alone has little or no value as related to 
performance implications in the future; one 
of the main purposes of resource and sched- 
ule analysis is to recognize implications and 
to reduce management surprises. This allows 
for identification and evaluation of “what ifs” 
and alternatives. The initial and subsequent 
cost estimates must recognize and quantify 
risks and uncertainties and provide reserves 
and allowances for program changes. The re- 
quirement for uncertainties and risk is as vi- 
tal to project success as any other cost ele- 
ment. Having contingency funds available 
and using them judicially are integral parts 
of successful research and development ef- 
forts. 

If the contractor reporting structure at- 
tempts to closely parallel schedule and cost 
reporting milestones, extreme care should be 
taken that it is not based on the assumption 
of equal value milestone performance. This 
type of system can easily lead to some mis- 
leading assessments. If such a system is 
used, program changes can completely dis- 
rupt performance reporting and require in- 
stallation of a new structure of report ac- 
counts and a long hiatus in reporting. To 
base a system on an assumption of continued 
program equilibrium would be a mistake — 
uncertainty is much more likely to be the 
norm. 
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Configuration Management 

The purpose of configuration management is 
to provide a disciplined systems approach for 
the control of the requirements and configu- 
ration (normally established by systems en- 
gineering) of hardware and software to be de- 
veloped and the process for change consider- 
ation.The function basically consists of four 
distinct practices: 

• Configuration Identification — The defini- 
tion and establishment of the total techni- 
cal requirements (performance and func- 
tional) and the detailed configuration 
definition and documentation. Configura- 
tion identification is usually established 
incrementally as design and development 
proceed. 

• Configuration Control — The formal pro- 
cess used to establish and control changes 
to the configuration baseline. This control 
is effected through a hierarchy of formal 
configuration control boards established 
at the different levels of hardware and 
software. 

• Configuration Accounting — Performance 
of this function “defines” the exact base- 
line on a continuing basis and provides a 
clear audit trail from the authorization of 
changes into the affected documentation. 
It should provide the single authoritative 
source for baseline definition. 

• Configuration Verification — Ensures that 
the baseline configuration requirements 
have been incorporated into contracts and 
are fabricated and tested accordingly. 

Documentation Management 
Documentation management establishes 
data policies and responsibilities and proce- 
dures for identifying, planning, selecting and 
scheduling a large volume of data. The data 
management system ensures continual man- 
agement review of NASA-generated and con- 
tractually required documents, eliminates 


any non-essential requirements, and assures 
only the minimum amount of documentation 
necessary for effective program manage- 
ment. The principal intention of the system 
should be to define the information required, 
justify its need, and control the information 
after it is generated. 

Schedule Management 
This function provides for the development 
and maintenance of the master schedule and 
the detailed, interrelated schedules covering 
the total program or project to completion. It 
involves the requirement to define the sched- 
ule format, content and symbols used.. A 
critical component of the function is selecting 
the key progress indices for measuring per- 
formance and indicating potential problems. 
A system of reports, reviews, and action feed- 
back needs to be provided. Working closely 
with resources management, the analysts 
must evaluate performance, synthesize var- 
ious inputs and implications, and generate 
and evaluate alternatives. Plans and sched- 
ules should provide for uncertainty and the 
unknown. 

The integrity, reliability and discipline of the 
reporting system are essential. NASA should 
continually assure the end-to-end integrity 
of program control data from its source in 
subcontractors to prime contractors and sub- 
sequent levels of NASA. The importance of 
early problem recognition cannot be overem- 
phasized: the ideal control system detects po- 
tential deviations before they become actual 
ones. The costliest aspect of a development 
program is time. Slippages in a program 
schedule are extremely expensive. A perma- 
nent record of all changes and slippages 
should be kept to allow trend analyses. 

The primary steps of management accounta- 
bility-establishing objectives and baselines, 
measuring performance against baselines, 
analyzing and evaluating performance and 
alternatives, assigning action or direction, 
and ensuring action feedback — are applica- 
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ble to the management of almost any activ- 
ity. Some aspects of control functions such as 
planning, scheduling activities, and manag- 
ing resources are also applicable in some de- 
gree on all NASA work, including applied re- 
search and technology, science tasks, and in- 
stitutional management. However, the col- 
lection and staffing of the full array of project 
control functions are not necessarily appro- 
priate for all activities within NASA. The 
style of management and types of controls re- 
quire tailoring to the particular objectives 
and problems of the individual activities. 

How program control functions are grouped 
organizationally is a consequence of a num- 
ber of factors. Nevertheless, it is clear that 
all of the functions and their outputs need to 
be integrated. On a small project, a project 
manager could possibly perform the func- 
tions and integrate the data output. On rela- 
tively large or complex development projects 
or programs it is the opinion of the Academy 
team that management control and synthe- 
sis of program element progress and perfor- 
mance are enhanced by grouping the func- 
tions. A model that lays out program control 
functions suitable for most large and com- 
plex development projects is shown in Figure 
1. This model assumes that program analysis 
is an inherent part of the functions shown. 
As a matter of preference, however, program 
analysis can be handled as a self-contained 
function. 

Current Status of Program Control 
in NASA 

As a part of this study, the Academy made an 
effort to ascertain the current status and 
health of program/project control functions 
and processes within NASA. Interviews and 
discussions were held in both Headquarters 
and Centers with Center Directors, directors 
of flight projects, program managers and per- 
sonnel who play roles in program control 
functions. Discussions were also held with 
previous NASA program directors, some 
aerospace industry officials and support con- 


tractors supplying management services to 
NASA. 

In Headquarters, the reinstitution of the Pro- 
gram Approval Document (PAD) System has 
not moved swiftly. Dale Myers, Deputy Ad- 
ministrator in 1987, sent a letter with 
instructions for preparation of PADs in June 
1987. On March 14, 1989, a management in- 
struction (NMI 71211) was issued, which re- 
quired the specific development of 23 sepa- 
rate PADs with provisions for adding or de- 
leting projects in the future. Approximately 
eight have been prepared and approved. The 
Deputy Administrator is holding meetings 
with program offices in an attempt to tailor 
the format, content and level of detail of the 
document, and to define the management 
processes to fit the desired methods of opera- 
tion in an orderly and efficient fashion. 

Since early in its history, NASA documented 
its management policy and principles of pro- 
ject management as well as instructions on 
planning and approving major research and 
development projects. These instructions 
were canceled in the mid-1980s when the 
PAD system was eliminated. Efforts have ap- 
parently been made to reinitiate or replace 
some of the canceled documents, but at this 
point, it has not been accomplished. An un- 
derstanding of how the Agency intends to op- 
erate and what is expected in terms of project 
management approaches and techniques 
does not currently exist within the Agency. A 
common concern among senior managers at 
the Centers was the apparent lack of appre- 
ciation of the usefulness of such policy state- 
ments on the Agency’s operation. 

General management status reviews contin- 
ue to be held at Headquarters. The current 
review system provides for three separate 
meetings— one for Space Transportation and 
Space Station, another for all other programs 
and projects, and a third for institutional ac- 
tivities. According to some attendees, these 
reviews could not be characterized as disci- 
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plined reviews of progress against estab- 
lished baseline milestones and goals. Howev- 
er, program offices participating in these re- 
views do characterize the status and prob- 
lems of projects. 

The organization and performance of pro- 
gram/project control functions within the 
program offices and the development Centers 
have not materially changed or improved 
since the Management Study Group findings 
in 1986 and 1988. There have been some 
changes in personnel and in the methods of 
performing the functions. One trend appears 
to be an increasing use of support contractors 
to provide some project control functions in- 
cluding scheduling, configuration manage- 
ment, data management, and elements of fi- 
nancial operations. The degree of contractor 


use varies among the Centers but the trend 
[in 1989] appeared to be growing throughout 
NASA in all functions and activities in addi- 
tion to program control. The impetus for con- 
tracting out functions was generally attrib- 
uted to the need for supplementing the limit- 
ed availability of civil servants. Discussion 
with one NASA support contractor, however, 
confirmed that contractors also had the same 
difficulties in finding skilled personnel in 
program control disciplines and were faced 
with a problem of how to train their people 
and how to sharpen their skills. 

In reviewing the list of program control func- 
tions with NASA Center personnel, the re- 
viewers found no disagreement that all of the 
program control functions were required and 
should be performed on development 


Develop and maintain integrated planning base 
and program requirements and development 
plans; establish baselines of content, scope, 
configuration, schedule and cost; measure 
performance against the baselines; analyze and 
evaluate performance and alternatives; provide 
and monitor the procedures for changing the 
baselines; and provide the system of reports, 
reviews and action feedbacks. 



Program 

Control 


Establish and 
maintain a system 
(baseline) for a 
series of develop- 
ment plans and 
technical require- 
ments, setting both 
the terms of 
accountability and 
performance 


Establish, monitor 
and maintain cost 
and manpower 
baselines 

• Establish 
reporting and 
statusing 
structure 

• Correlate with 
schedule and 
performance 

• Identify and 
evaluate "what 
ifs" and 
alternatives 


Establish and 
maintain schedule 
baseline 

• Format and 
hierarchy of 
interrelated 
schedule 
covering total 
program 

• System of 
reports and 
review 

e Analyze and 
evaluate 
performance 
and alternatives 


Establish and 
maintain a uniform 
system of 
documentation 


Formal and 
disciplined system 
for tne establish- 
ment and control 
of baseline require- 
ments and config- 
urations of hard- 
ware and software 

• Configuration 
identification 

e Configuration 
control system 

e Configuration 
accounting 

e Configuration 
verification 


Figure 1. Program Control Functions 


7 











READINGS IN PROGRAM CONTROL 


projects. Only two organizations had essen- 
tially all of the program control functions op- 
erating together in one group. At Goddard 
Space Flight Center the functions were all 
within the Project Director’s office reporting 
to the Deputy Director for resources. Sched- 
uling, configuration management and data 
management functions were performed by a 
support contractor and were under civil ser- 
vice monitors responsible for the functions. A 
discrete function for project planning was not 
within the project offices. The Space Station 
office at Johnson Space Center (JSC) is the 
other organization having a fairly complete 
grouping of functions under the program con- 
trol division. In the other program offices at 
JSC, program control functions are not inte- 
grated in one group but are being performed 
in one way or another in various organiza- 
tions. 

At the Lewis Research Center, steps have 
been taken in the Space Station project to in- 
tegrate resource management, scheduling, 
and configuration management in a program 
control organization. At the Marshall Space 
Flight Center, there is a fairly consistent 
pattern of combining scheduling and re- 
sources management in a single organization 
in the project offices. Except for the cases 
noted above, the remainder of the NASA 
Centers and the Headquarters program of- 
fices do not have organizationally integrated 
program control functions. The functions are 
either not performed, are scattered in var- 
ious subgroups, or are done informally. 

An Agency cost estimate is always prepared 
on new development projects prior to evalua- 
tion and selection of contractors. However, 
there does not appear to be a uniform proce- 
dure for recycling and validating new esti- 
mates after selecting the development con- 
tractor. Rather, the contractor’s negotiated 
bid generally becomes the baseline against 
which any changes are incrementally made. 
This is true even though the contractor’s esti- 
mate is usually considerably lower than the 


government’s estimate. The rationale for the 
government’s higher estimate in most cases 
is quickly forgotten. Credibility begins to be 
attached to the contractor’s estimate, which 
is neither justified nor borne out by history. 
Since it takes some time for deficiencies to 
become apparent they generally come as sur- 
prises and result in more costly schedule slip- 
pages. In too many cases a large proportion of 
the time available to the staff of project re- 
source and schedule management groups is 
spent on finding near-term funding solutions 
to these “surprises.” 

As a general observation, too little effort is 
spent by both resource and schedule groups 
in analyzing potential problems or risks and 
in selecting critical reportable milestones 
that could give some advance notice on the 
probabilities of problems. Close correlation of 
reportable schedule and cost performance 
data is desirable, but the critical indices of 
performance are not always precisely aligned 
with a hardware-driven work breakdown 
structure. 

There is an apparent lack of emphasis on lay- 
ing out logic diagrams or networks on pro- 
jects, particularly prior to selecting schedule 
and resource reporting items. The research- 
ers know of no better way to comprehend in- 
terrelationships and interfaces of efforts on 
components, subsystems and systems. When 
these networks are laid out in time sequence, 
critical schedule and resource reporting indi- 
ces become much more apparent and risks 
are easier to assess. The special virtue of log- 
ic diagrams is that they allow planners to in- 
corporate time, resources, and technology 
into strategies, thus linking temporal hori- 
zons with contextual changes. 

As a generalization, reviews at the Centers 
appear to be more structured toward the as- 
sessment of project performance than they 
are in Headquarters. Many of the reviews at 
the contractor plants, however, seem to be 
primarily scheduled visits with fixed agen- 
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das, and with large groups spending great 
amounts of time looking at viewgraphs. It 
was not apparent how often site visits by pro- 
ject control personnel were made for the pur- 
pose of assessing performance and verifying 
the integrity of reported data at its source. 
Regardless of how scientific the approach or 
how sophisticated the management system 
and tools are, there is no substitute for a sim- 
ple visual assessment. Coordination of those 
supplying performance data is essential. 

Training 

Traditionally within NASA, program control 
personnel have gained skills and knowledge 
through first-hand experience and from their 
experienced supervisors. Immersing them- 
selves in program/project research and devel- 
opment activities is still the most common 
way of gaining project management knowl- 
edge. Forming mentor relationships — work- 
ing with a person who can provide counsel- 
ing, guidance and advice — is also used to 
gain the skills and credentials of program 
control. However, experienced program con- 
trol personnel are becoming fewer within 
NASA. According to interviewees at the God- 
dard Space Flight Center, in the past, many 
program control staff first studied operations 
research or industrial engineering, then ac- 
quired on-the-job skills and subsequently 
passed on lessons learned by various means. 
Rarely did program control staff receive for- 
mal training related to specific functions 
such as the establishment and maintenance 
of a configuration control or scheduling sys- 
tem. 

NASA and contractors currently face diffi- 
cult problems in recruiting experienced pro- 
gram control staff due to a number of rea- 
sons, from limited career paths to elimina- 
tion of industrial engineering disciplines at 
many major universities. As mentioned ear- 
lier, in response to recommendations from 
the Phillips Committee, NASA decided to 
formalize efforts to help in the development 
and training of managers, including program 


control personnel. Formal training will be 
provided in such areas as resources manage- 
ment, schedule management, and configura- 
tion management. Analytical skills and the 
philosophical and logical foundations of pro- 
gram control, however, cannot be learned 
just by attending classes. They require appli- 
cation and the achievement of an end result 
as well. Self organization, program interest, 
ability to coordinate individuals and data, a 
questioning attitude, resiliency, sensitivity, 
imagination, and practicability are other 
nonempirical qualities that are valuable in 
program control work, but are beyond the 
realm of classrooms. In sum, formal courses 
can only complement, not replace, hands-on 
experience and the inherent qualities of key 
personnel. This is because analytical skills 
are, to a large extent, embodied in people and 
institutions, not just in physical objects like 
computers. 

It is anticipated that formal development 
training will be provided by both civil ser- 
vants and contractors. There will be a core 
curriculum which will be designed to serve 
business, technical and program and project 
management staff as well as a series of de- 
tailed courses designed for people who will be 
performing functions in specific areas. It is 
expected that the importance of integration 
of the program control functions and synthe- 
sis of data, personal responsibility and ac- 
countability, and disciplined procedures will 
be stressed. How the courses are structured 
and how consistent they are with the past ex- 
periences and needs of trainees will have a 
strong bearing on the prospects for success of 
the training efforts. Equally important, how- 
ever, will be the support of top management 
at the Centers and Headquarters. Their in- 
terest will have a serious impact on the out- 
come of the project. If top management is 
sensitive to and supportive of the need for 
training and displays a strong commitment 
to the training program, the probability of 
success increases tremendously. Perhaps 
more significant is that if top management is 
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involved and accurately communicates its in- 
volvement, the entire effort will be perceived 
as credible and worthwhile. 

Recommendations and Observations 
NASA has successfully managed some of this 
country’s most complex technology and de- 
velopment programs. These successes have 
included the application of sound program 
control processes. The basic concepts of pro- 
gram management and program control 
have not changed, although computerized 
systems have the capability to enhance the 
quality and effectiveness of documentation, 
communications, evaluation tools and sup- 
port systems. Much of the new capability of 
tools and support systems have been incorpo- 
rated in NASA, but over time NASA’s use of 
the basic management control disciplines 
has weakened. Strengthening program con- 
trol involves the improvement and utiliza- 
tion of certain disciplines, the existence of a 
conducive Agency environment and an un- 
derstanding throughout the Agency of the 
leadership’s policy and objectives. The fol- 
lowing recommendations are oriented to- 
ward improvements in program control pro- 
cesses and practices. 

Enhancement of Agency Environment 
for Effective Program Control 
This study concludes that it would be ex- 
tremely helpful for NASA personnel to be 
aware of the importance attached to program 
control functions by the Office of the Admin- 
istrator. This awareness can result in the re- 
invigoration of program management disci- 
plines throughout the Agency. An effective 
method of informing Agency personnel and 
contractors would be through appropriate is- 
suances setting forth Agency intentions for 
conducting its business, expectations of all 
elements and policies and procedures for pro- 
gram/project approvals, assignment of re- 
sponsibilities and the explicit accountabil- 
ities of organizations and individuals. 

The following actions would be helpful: 


• Issuance of Agency policies and processes 
for the approval and conduct of projects, 
the assignment of responsibilities and the 
terms of accountability of organizations 
and personnel. 

• Establishment of regular performance re- 
views against approved baselines of de- 
velopment plans, schedules and cost ap- 
propriate for this level of management. 

• Facilitation of rapid communications to 
and from all NASA elements regarding 
program control functions, tasks and 
feedback on action assignments. 

Development of Training Activities for 
Program Control 

The primary emphasis should be on under- 
standing the role of program control func- 
tions in relation to and in context with the 
program/project manager and other groups 
and functions of the program office, particu- 
larly systems engineering. Systems engi- 
neering includes those activities required to 
transform mission needs into a comprehen- 
sive and definitive set of systems perfor- 
mance requirements. It also includes the ac- 
tivities needed to define a preferred systems 
configuration and its detailed performance 
requirements. The results of these activities 
set much of the baseline detail for program 
control functions including program plans 
and configuration management and param- 
eters for schedule and cost management. 

Program control is the total management 
process of establishing and maintaining the 
official development plans and program 
baselines in a manner which maximizes suc- 
cess in meeting a program’s overall objec- 
tives. Although the following topics are not 
all-inclusive, some suggested program con- 
trol training activities are (more detail 
shown in Appendix A): 

1. Philosophy, content and context of ef- 
fective program/project control. 
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2. Planning and documentation of re- 
quirements. 

3 . Content and processes of configuration 
management. 

4. Logic diagrams or networks. 

5. The scheduling function and process. 

6. Basics of primary methods of cost 
estimating. 

7. Resource management and control. 

8. Presentation of data. 

The most important element in evaluating 
and assessing the status of a project and pro- 
viding program control is the understanding 
of the objectives, technical content, develop- 
ment approach, and the interrelationships 
and interfaces involved in its development. 
Throughout this report, the Academy re- 
searchers have taken the position that pro- 
gram control is not a collection of the sepa- 
rate functions that comprise it, but that it is 
an understanding of the plans and approach 
and the interrelationships of the functions 
and performance of configuration, schedule 
and resource management. 

The most meaningful implications from per- 
formance data cannot be drawn from the in- 
dependent functions, but rather, only when 
the data are integrated. For this reason we 
have emphasized the integrated understand- 
ing of the roles rather than skills and tools. 
Tools and skills can be very important but 
only when one understands their limitations 
as well as advantages and knows when they 
can be usefully applied. In this context, em- 
phasis on skill training is important with re- 
gard to particular tasks such as logic net- 
works, a means of focusing data for the maxi- 
mum information output and the presenta- 
tion of interrelated performance data. 


Observations 

Until conducting this study it had not been 
apparent to the researchers the degree to 
which NASA has become staffed with sup- 
port contractors as opposed to career civil 
servants. On-site contractors appear to now 
exceed civil servants. The impact of this con- 
dition potentially can have serious conse- 
quences on NASA’s program management 
and control capabilities. 

As stated earlier in this report, NASA pro- 
jects push technology beyond the current 
state of the art. Traditionally NASA has had 
the civil service and fabrication capability in 
its centers to conduct the appropriate depth 
of studies, examine objectives and missions, 
develop the technical concepts for accom- 
plishing missions, determine feasibility, and 
provide the conceptual design. If it was decid- 
ed to budget and contract for the design and 
development of a project, the inhouse capa- 
bility existed to manage, technically moni- 
tor, evaluate, and direct such contracted 
work. If technical problems arose at the con- 
tractors’ plants, the capability existed to help 
provide solutions and correct the problems. 
Some of the major objectives of pro- 
gram/project control are the early identifica- 
tion of potential problems, avoidance of sur- 
prises, provision of workarounds, and the 
ability to obtain help in providing solutions. 
This precept of the importance of early prob- 
lem identification assumes the availability of 
the technical capability to participate in so- 
lutions to such problems. 

Funding pressures on projects have contin- 
ued since the early 1970s and less funding al- 
lowance for the contingencies of the “un- 
known” has been the result. As surprises oc- 
cur and additional funds are not available, 
schedules usually become the variable on 
which short-term solutions to fiscal year fun- 
ding problems are based. The obvious result 
is an increased run-out on total cost and 
shrinking credibility. 
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With the increasing contractor staffing, 
NASA engineers have less and less “hands- 
on” experience. Service contractors are in- 
creasingly being used at Centers to perform 
project control functions such as scheduling, 
configuration management and elements of 
financial management. In effect, this is using 
contractors to monitor the performance of 
prime development contractors. This situa- 
tion is leading some NASA managers to 


question the Agency’s continuing ability to 
manage contracted projects and control costs. 

NASA remains responsible for the perfor- 
mance of the work, but with a reduction in 
capability to influence and correct perfor- 
mance. How well the Agency meets demands 
relating to program performance has a major 
effect on its ability to effectively run pro- 
grams. 
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Appendix 

Suggested Training Activities 

The following topics are not inclusive in the 

sense that they cover all items. 

1 . Philosophy, content and context of 

effective program/project control. 

- What is meant by “control”? 

- An explanation of how the main functions 
relate to each other. 

- Importance of understanding the totality 
of the project. 

- Importance of understanding 
interrelationship of elements and 
interfaces. 

- Importance of ensuring integrity of 
reported data to source level. 

- Importance of concentrating on the impli- 
cations of reported data rather than on 
the factual data. 

- Anticipation of development difficulties 
and changes in external environment. 

- Continual assessment of “what ifs.” 

- Importance of a questioning approach. 

- Requirement for disciplined processes and 
positive monitoring. 

- Barriers to effective program control. 

2. Planning and documentation of require- 
ments. 

- Importance of maintaining development 
plan baseline. 

- The necessity of a series of subsidiary 
plans, actions, and schedules. 

- Documentation of requirements. 

- Technical and program reviews and re- 
sults. 

3. Content and processes of configuration 

management. 

- Importance of early development and doc- 
umentation of configuration require- 
ments and preparation of a configuration 
maintenance plan. 

- The systematic approach of defining and 
documenting the detailed configuration. 
Understanding of the need for increment- 


al identification as design and develop- 
ment proceed. 

- The significance of positive control of 
changes to configuration. Importance of 
evaluating impact of individual proposed 
changes on operational capability and to- 
tal cost. 

- Importance of clear audit trail of changes 
and maintenance of the exact baseline. 

- Necessity for effective verification that 
baseline configuration has been imple- 
mented. 

4. Logic diagrams or networks. 

- Understanding how to develop networks. 

- Importance for understanding the total 
job and the interrelationship of the com- 
ponents of the job. 

- Relevance of networks to effective analy- 
sis and synthesis of performance data. 

5. The scheduling function and process. 

- Critical importance of identifying known 
and potential development risks. 

- Planning for the unknown. 

- Understanding interrelationships and in- 
terfaces of development processes and or- 
ganizations. 

- Importance of selecting the critical indica- 
tors of progress or problems - the most im- 
portant scheduling function. 

- Identification of indicators as far “up- 
stream” as possible from critical progress 
points. 

- The danger of becoming mesmerized with 
systems. The need to understand weak- 
nesses better than positive elements and 
to keep systems as simple as possible. 

- The amount of time required for adminis- 
trative and decision processes. This time 
requirement cannot be overlooked. 

- The costliest aspect of a R&D project is 
time. Slippages are extremely expensive. 

- Emphasis on early problem recognition. 

- Importance of having only authenticated 
and dated schedules. 

- Maintenance of permanent record of all 
changes and documentation of slippages. 
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6. Basics of primary methods of cost esti- 
mating. 

- Understanding of concepts, processes, 
when each is most useful, advantages and 
disadvantages: 

• Parametric cost estimating; 

• Analogy estimates; 

• Engineering estimates (“grassroots”, 
“bottom up”); and 

• Expert opinion or Delphi techniques. 

- Dangers of accepting contractor’s negoti- 
ated cost estimate without complete re- 
verification. 

- Importance of quantifying risks. 

- Importance of provision for and use of 
reserves. 

- Risks involved in using cost goals as in- 
centives in cost estimating and the use of 
“design to cost” concepts on R&D oper- 
ational systems. 

7. Resource management and control. 

- Establishment a cost reporting system. 

- Importance of correlating manpower re- 
ports on R&D projects. 

- Importance of integrating cost data with 
schedule performance. 

- V erification of end-to-end integrity of 
data reported. 

- Understanding the contract structure, 
and nuances of differences in definitions 


and accumulation processes of prime and 
subcontractors. 

- Importance of on-site verification of data 
and calibration of personnel supplying 
data. 

- Reporting of data should raise questions, 
not answer them. 

- Trend analyses. 

- Emphasis on run-out and cost to comple- 
tion. 

- Importance of continual work on “what 
ifs.” 

8. Presentation of data. 

- Determination of objective or purposes of 
presentation: What is the message or in- 
formation to impart? 

- Determination of desired outcomes. 

- Avoidance of reams of cost, schedule or 
engineering data. The need to focus pre- 
sentations and use only data which contri- 
bute to understanding context, signifi- 
cance and implications of information. 
Detail can overwhelm strategic choices. 

- Factual data may or may not be signifi- 
cant to future actions or decisions even 
though they may be important for legal or 
audit purposes. 

- The need to sequence messages in a prior- 
ity, logical or temporal order. The use of 
unambiguous language. 
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Cost Estimating Methods for Advanced 
Space Systems 

by Kelley Cyr 


NASA is responsible for developing much of 
the nation’s future space technology. Cost es- 
timates for new programs are required early 
in the planning process so that decisions can 
be made accurately. Because of the long lead 
times required to develop space hardware, 
the cost estimates are frequently required 10 
to 15 years before the program delivers hard- 
ware. The system design in conceptual 
phases of a program is usually only vaguely 
defined and the technology used is so often 
state-of-the-art or beyond. These factors com- 
bine to make cost estimating for conceptual 
programs very challenging. 

This paper describes an effort to develop 
parametric cost estimating methods for space 
systems in the conceptual design phase. The 
approach is to identify variables that drive 
cost such as weight, quantity, development 
culture, design inheritance and time. The na- 
ture of the relationships between the driver 
variables and cost will be discussed. In par- 
ticular, the relationship between weight and 
cost will be examined in detail. A theoretical 
model of cost will be developed and tested 
statistically against a historical database of 
major research and development projects. 

Cost Theory 

In order to meet the needs of NASA for a 
long-range forecasting tool, the following re- 
quirements were laid down: 

• Must have the ability to predict cost over 
long time horizons (25 to 50 years). 

• Must be valid for substantially different 
of systems. 

• Must be able to predict cost reliability de- 
spite significant technological advances. 

• Require few inputs and be simple to use. 


In order to determine the feasibility of a 
model that would meet the specified require- 
ments, a proof of concept test was devised. A 
theoretical model was developed for predict- 
ing the total acquisition cost of a major hard- 
ware development program. The variables 
used in the model are described below. 

Quantity Variable. The relationship be- 
tween the quantity or number of units pro- 
duced can take many forms. In Figure 1, four 
of the most common forms are illustrated. 
Figure la illustrates the unit or average cost 
method in which the average cost per unit is 
used. In this case, the average cost is the 
same regardless of the quantity produced. 
This method is most useful for small quanti- 
ty buys of commercial products where the 
quantity purchased does not materially af- 
fect the cost of production. 



A second method of estimating cost, illustrat- 
ed on Figure lb, is the fixed plus variable 
cost method. The marginal cost, in this case, 
is constant. The average cost is higher than 
the marginal cost, decreases as the quantity 
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increases and approaches, but never reaches, 
the marginal cost. In this case, the fixed cost 
is relatively large and changing the quantity 
produced can substantially affect the aver- 
age cost. This model represents increasing 
economies of scale. 

The third method, illustrated in Figure lc, 
incorporates the principle of decreasing mar- 
ginal cost. In other words, the additional cost 
of each unit is slightly less than the previous 
unit. This principle is also known as the 
learning curve or experience curve. The 
learning curve also has decreasing average 
unit cost as the quantity is increased. 

A fourth type of quantity relationship is 
shown in Figure Id. In this case, the margin- 
al cost increases for the first several units, 
then begins to decrease along the lines of a 
learning curve as quantity increases further. 
This example would represent a situation 
where the first few units were partially oper- 
ational or low cost prototypes were gradually 
building up to full scale production articles. 
Once a reproducible configuration is reached, 
the marginal cost decreases according to 
learning curve principles. 

Weight Variable. Weight has been used for N 
many years in estimating the cost of aero- 
space systems. It is a most convenient vari- 
able since it generally characterizes the size 
and often, the performance of a piece of hard- 
ware. Weight is also a key engineering pa- 
rameter; therefore, an estimate of it is usual- 
ly available, even at the early stages of a pro- 
gram. Although the emphasis here is on 
weight, the discussion could also be applied 
to other descriptive parameters such as size, 
speed, power, etc. 

The following discussion refers to weight as 
the dry mass of a single unit. Like quantity, 
weight can be related to cost in several ways. 
The most common relationships are depicted- 
in Figure 2. 


The simple cost-per-unit weight relationship 
is illustrated in Figure 2a. By definition, the 
cost-per-unit weight model has constant 
average cost-per-unit weight. 



Figure 2. Total Cost Versus Unit Weight 


The model in Figure 2b has the characteris- 
tic fixed plus variable cost. In this case, the 
average cost per unit weight decreases as the 
weight increases. The marginal cost is con- 
stant and average cost is asymptotic to mar- 
ginal cost. This is a case of economies of scale 
with respect to unit weight. 

Figure 2c illustrates a model in which the 
marginal cost is decreasing; hence, the aver- 
age cost is decreasing. In this case, the rate of 
change in the marginal cost is also decreas- 
ing. 

The total cost relationship shown in Figure 
2c is an exponential growth function. The ex- 
ponent happens to have a special meaning in 
economics: it is the elasticity of cost with re- 
spect to weight. If the elasticity is greater 
than 1, then the relationship is said to have 
decreasing economies of scale. If the elastic- 
ity is greater than 0 but less than 1, then 
there are increasing economies of scale. If the 
elasticity is exactly 1, then there are con- 
stant economies of scale. 
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Clearly, if there are strong economies of 
scale, it would be better to build larger 
(heavier) things. It should be noted, however, 
that weight and quantity may also be relat- 
ed. The larger something is, the less likely it 
is to be built in large quantities. The rela- 
tionship between cost and quantity may also 
have economies of scale; therefore, the effect 
of different weights on both cost and quantity 
should be considered when estimating total 
program cost. 

In the last case, Figure 2d, the marginal cost 
weight is negative up to a certain weight, 
then becomes positive. The total cost curve 
becomes U-shaped (also known as a bucket 
curve). This curve represents a situation 
where there is an optimum weight for a giv- 
en type of hardware. Any attempt to decrease 
the weight below optimum would require ad- 
ditional cost through the use of exotic mate- 
rials, additional manufacturing processes, or 
more complex fabrication techniques. By the 
same token, attempts to increase the weight 
above optimum would require additional cost 
for high performance propulsion, additional 


structural analysis and testing, specialized 
tooling, et cetera. 

Culture Variable. So far, it has been postu- 
lated that significant relationships exist 
among cost, quantity and weight. It is not 
likely, however, that the relationships are 
exactly the same for all different types of 
hardware. A situation, such as the one in 
Figure 3, may exist where the cost versus 
weight curves for several types of hardware 
have the same elasticity but different multi- 
pliers. The culture variable is defined as a 
value representing the vertical height of the 
cost/weight curve for a given subcategory of 
hardware. If the cost weight curves were 
plotted on a log-log graph, the lines would be 
parallel straight lines and the culture vari- 
able would be a function of the vertical inter- 
cepts. 

A category is defined as a group of hardware 
systems that are functionally similar; such 
as, aircraft, ship, or spacecraft. A subcatego- 
ry describes a group of systems that perform 
a similar mission or have the same oper- 


TOTALCOST 



UNIT DRY WEIGHT 


Figure 3. Culture Variable 
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ational environment. The subcategories of 
aircraft would include fighter, bomber, 
transport, etc. The classifications used in this 
paper are listed in Table 1. 

It must be assumed, for the convenience of 
regression analysis, that the elasticities are 
the same for all subcategories. This will 
prove to be an overly restrictive assumption, 
and future work may focus on techniques to 
eliminate the need to make it. 

Complexity Variable. Within a given sub- 
category, it is possible that the systems may 
vary considerably in terms of performance, 
capacity, level of technology, complexity of 
design, and many other factors. Variations of 
the type listed within a given subcategory 
are henceforth referred to by the variable 
name complexity. Complexity is obviously 
very difficult to define and quantify a priori. 


The potential for overlap between culture 
and complexity can also create confusion. Re- 
search and development organizations tend 
to group along functional and mission lines; 
the classification scheme used for culture in- 
herently contains organizational informa- 
tion as well. Organizational differences with- 
in a given subcategory may be included in 
complexity. Also, specification levels vary 
along the functional lines in platform, so 
only the specification differences within an 
established subcategory should be considered 
in complexity. 

Since there is no readily available means of 
quantifying complexity a priori, this variable 
will not be used in the subsequent model 
derivation. It is discussed here in order to 
clarify the definition of culture and to pro- 
vide a basis for future work to refine quanti- 
tative measures of complexity. 


Table 1. Culture Classification Scheme 


SUBCATEGORY 

NO. 

CULTURE 

SUBCATEGORY 

NO. 

CULTURE 

AIRCRAFT 

63 

1.82 

SHIPS 

29 

1.14 

ATTACK 

8 

1.96 

A/C CARRIERS 

5 

1.11 

BOMBERS 

8 

1.99 

AMPHIB. ASSAULT 

5 

0.89 

FIGHTERS 

16 

1.94 

CRUISERS 

4 

1.19 

FW-TRANSPORTS 

10 

1.63 

DESTROYERS 

5 

1.25 

PATROL 

5 

1.88 

FRIGATES 

3 

1.14 

ROTARY ATTACK 

5 

1.88 

SUBMARINES 

7 

1.24 

ROTARY CARGO 

5 

1.75 




TRAINER 

3 

1.46 

GROUND MOBILE 

16 

1.15 

COMMERCIAL 

3 

1.74 

TANKS 

4 

1.24 




TRUCKS 

7 

0.82 

MISSILES 

87 

1.89 

APCs 

2 

0.96 

AIR-AIR 

13 

2.04 

RIFLES 

3 

1.59 

AIR ORBIT 

1 

2.04 




AIR SURFACE 

15 

1.81 

SPACECRAFT 

56 

2.18 

ANTI-TANK 

4 

1.78 

MANNED REENTRY 

5 

2.34 

ICBM 

11 

1.92 

MANNED ORBITAL 

2 

2.05 

SURFACE-AIR 

12 

1.97 

PHYSICS & ASTRONOMY 

12 

2.20 

SHIP AIR 

9 

1.74 

EARTH OBSERVATION 

6 

2.04 

SURFACE-SURFACE, LAND 

8 

1.88 

WEATHER 

7 

2.19 

SURFACE SURFACE, OTHER 

4 

2.07 

COMMUNICATION 

9 

2.22 

ICBM (SUB) 

4 

1.89 

MISC. SPACE 

2 

1.95 

ROCKETS 

6 

1.64 

PLANETARY 

13 

2.45 




UNMANNED REENTRY 

7 

2,04 
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Time Variable. Another factor that must be 
considered in estimating cost is the impact of 
time-related phenomenon. Inflation, produc- 
tivity, technology and performance are just a 
few of the factors that may change with time. 
For most cost estimating applications, the ef- 
fects of inflation are removed by applying 
standard inflation rates to convert the data 
to a constant-year dollars. The modeling of 
productivity, performance and technology 
change is not so easy. 

Time-related phenomena may change at a 
fixed rate, like interest on a bond, or they 
may vary from one time period to another. 
The method of using a program milestone 
date as the time variable will result in a 
fixed rate of change when the model is esti- 
mated. Measurement of the variable rate 
case would require construction of an index, 
similar to an inflation index, and then select- 
ing the appropriate index value based on the 
year of Initial Operational Capability (IOC), 
mid-point of construction or some other ba- 
sis. A productivity or technology improve- 
ment index could be incorporated in this 
fashion. For this report, the IOC year was 
chosen to represent time. 

Generation Variable. The design of a new 
aircraft, spacecraft or missile is often based 
on a previous design that has already been 
proven. A new airplane may use the previous 
airframe with only minor structural modifi- 
cations. Spacecraft designs may use structur- 
al components, electronics, and mechanical 
systems already tested on a previous design. 
Designers may work with configurations 
they are familar with from previous projects. 
The result may be considerable savings in 
the development cost of new hardware. Sav- 
ings can also be achieved in production since 
the tooling already exists and manufacturing 
experience is far down the learning curve 
from the previous design. 

In theory, the cost of each subsequent model 
should be considerably less than the previous 


model. The amount of savings, however, 
would probably decrease as the series pro- 
gresses. The total cost would be decreasing 
asymptotically to some level as shown in 
Figure 4. 



The generation variable used in this paper is 
defined as the sequential number for a given 
model of a specific piece of hardware. Gen- 
eration is not used to represent individual 
units of production, but rather a group of 
identical units. Subsequent generations 
must have very similar characteristics usu- 
ally being produced by the same manufactur- 
er or to the same specifications. Individual 
units of production may be given a genera- 
tion number if they differ substantially from 
previous units but still retain the basic de- 
sign and total production is small. All pro- 
grams that do not have readily identifiable 
predecessors are given a generation of one. 

Statistical Analysis 

In order to statistically validate some of the 
theories relating to cost behavior, it was nec- 
essary to construct a database of cost and 
other variables for many different types of 
research and development programs. The da- 
tabase consists of 264 major programs. Most 
of the programs are U.S. Government spon- 
sored. Many of the Government programs 
are defense-related weapons and delivery 
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systems. A substantial number of NASA 
sponsored spacecraft are also included. A 
small proportion of the data comes from oth- 
er Government agencies, foreign countries 
and commercial companies. In total, the da- 
tabase represents $ 1 trillion worth of expen- 
ditures in 1987 dollars. 

Programs from the 1930s all the way up to 
the mid-1980s are included. Major categories 
include ground vehicles, ships, aircraft, mis- 
siles and spacecraft. Data collected for this 
study included top-level cost data, system 
weights, program schedule dates, developing 
organizations and technical data. A variety 
of sources were used to gather data, and in- 
formation was confirmed by two or more 
sources whenever possible. 

Model Evaluation. Model evaluation has 
consisted of three major steps. The first step 
was to test a model consisting of the vari- 
ables quantity, weight, culture, IOC year 
and generation against the database as a 
whole. Step 2 required the estimation of 


models for individual subcategories of data. 
Finally, the elasticities derived from step 2 
were compared to the culture variable de- 
rived in step 1. 

Step 1 had several major functions. One was 
to evaluate the theoretical model of quantity, 
weight, culture, IOC year and generation. A 
second function was to produce estimated 
values of culture for different program subca- 
tegories. A third purpose was to identify any 
data observations that may be incorrect or 
classified wrongly. The final function was to 
develop estimates for the elasticities of 
weight and quantity, as well as other pre- 
sumed constants. 

Using total program cost, weight, quantity 
and other data, a multiple linear regression 
analysis was performed. The results are pre- 
sented in Table 2. Out of 264 data points, 253 
observations were included in the regression 
model. The remaining observations were re- 
jected due to missing data. The dependent 
variable is the log 10 of total cost. The inde- 


Table 2. Regression Model Results 


Dependent Variable: Logic of Total Acquisition Cost 

Independent Variables: 

COEF. 



STD. 


VALUE 


T-STAT 

ERROR 

Constant 

-4.7645 




Q Logio Total Quantity 

0.5773 


47.5 

0.0122 

W Logio Unit Dry Weight (lbs.) 

0.6569 


43.5 

0.0151 

C Culture 

1.7705 


31.8 

0.0556 

Y IOC Year -1900 

0.0124 


9.3 

0.0013 

G Generation 

-0.3485 


-7.5 

0.0466 

Standard Error of Y Estimate 


0.2247 



R Squared 


0.9125 



Observations 


253 



Degrees of Freedom 


247 



MAPE 


45% 



0.5773 0.6569 C 


Y 

G 


COST= 0.00001 72Q W 58.95 

1.0291 

0.4483 
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pendent variables are log 10, weight log 10, 
total quantity, culture, IOC year and genera- 
tion. The coefficient of determination (R 
squared) is 0.91 and all of the variables are 
significant according to their statistics. Also, 
the signs and the magnitude of the coeffi- 
cients are reasonable. 

As discussed earlier, the culture variable is a 
derived value. The derivation begins by en- 
tering an estimated value for each culture 
subcategory. The multiple regression is per- 
formed using the original value for culture. 
The estimation errors for each subcategory is 
then adjusted by a factor calculated to make 
the average error for that subcategory zero. 

A new multiple regression is then performed 
with the adjusted culture values. This pro- 
cess is repeated until the regression statistics 
stabilize. In order to minimize rounding er- 
rors, culture values are rounded at the sec- 
ond decimal place prior to the regression 
analysis. 

A second regression analysis was done at the 
subcategory level for a few selected subcate- 
gories. This process generally used log 10 to- 
tal cost as the dependent variable and log 10 
weight and log 10 quantity as independent 
variables. In some cases, IOC year and gen- 
eration were also included. The results of 


step two are summarized in Table 3. Note 
that the R-squared values are good for al- 
most all subcategories. The elasticity of 
weight and elasticity of quantity are dis- 
played along with estimated culture values. 


WEIGHT 

ELASTICITY 






2 

1 

X 

X 






X X 

X 



0 




* X X 

X 

X 



0.5 

1.5 

2.5 




CULTURE 





Figure 5. Weight Elasticity Versus Culture 


The final step in the analysis was to compare 
the culture values to the elasticity values 
with respect to weight. Recall that culture is 
a function of the intercept of the regression 
lines and the elasticity is the slope of the re- 
gression lines in a log- log model. A regres- 
sion analysis of the dependent variable 
weight elasticity and the independent vari- 
able culture found high correlation with an 
R-squared of 0.80, or 0.95 with the one out- 
lier removed (see Figure 5). 


Table 3. Subcategory Model Results 


CAT 

SUBCATEGORY 

CULTURE 

WEIGHT 

Elastic. 

QUANTITY 

ELASTIC. 

R2 

S/C 

PLANETARY 

2.52 

0.45 

1.02 

0.87 

S/C 

PHYSICS & ASTRONOMY 

2.31 

0.68 

1.17 

0.95 

MSL 

AIR- AIR 

2.06 

0.69 

0.53 

0.86 

MSL 

ICBM 

1.97 

0.81 

0.92 

0.93 

A/C 

ATTACK 

1.97 

0.43 

0.52 

0.92 

A/C 

FIGHTER 

1.96 

0.74 

0.46 

0.95 

MSL 

AIR-GROUND 

1.89 

0.91 

0.57 

0.81 

A/C 

TRANSPORT 

1.68 

0.91 

0.54 

0.87 

SHIP 

SUBMARINE 

1.33 

1.18 

0.92 

0.99 

SHIP 

AMPHIB. ASSAULT 

0.98 

1.30 

0.30 

0.95 
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Furthermore, the coefficient of culture has a 
negative sign. This can be interpreted econo- 
mically as meaning that high culture pro- 
grams have greater economy of scale with re- 
spect to weight than low culture programs. 
Figure 6 illustrates the effect of the latter 
conclusion on the cost/weight curves. Note 
that moving down to the right increases the 
slope. 

It is also noteworthy that two subcategories, 
submarines and amphibious assault ships, 
actually had weight elasticities greater than 
one, indicating diseconomies of scale. 

An attempt was also made to correlate cul- 
ture with quantity elasticity but the results 
were inconclusive. Of particular interest are 
the quantity elasticities of planetary, physics 
and astronomy satellites which are 1.02 and 
1.17 respectively. The fact that these elastic- 
ities are close to or greater than one indicates 
that the marginal cost is constant or increas- 
ing. Since spacecraft generally have very 
small production runs, and the first few units 
are generally prototypes or test articles, this 
is not surprising. The high elasticities may 


be indicative of the S-curve depicted in Fig- 
ure Id. 

Model Validation. A procedure was devel- 
oped for validating the statistically estimat- 
ed model. At the time this paper was written, 
only the phase one total database model has 
been tested. The validation procedure con- 
sisted of dividing the database into two parts. 
The data was divided at the median IOC 
year, 1969. All programs prior to 1970 were 
used to calibrate a new model using the same 
variables as the overall model. Values for 
culture were also calibrated based on the 
limited data. 

The restricted model was then used to simu- 
late a forecast of the actual programs in the 
second half of the database. The result was 
that the simulated forecast overestimated 
the total actual cost by approximately 45%. 
This indicates a bias in the estimating 
model. An examination of the coefficients 
showed that all were reasonably consistent 
between time periods except for the coeffi- 
cient of IOC year. The coefficient for IOC 
year is 50% higher in the first period than 



10 100 10,000 1,000,000 10,000,000 

UNITDRYWEIGHT 

Figure 6. Subcategory Models 
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overall. This difference probably accounts for 
most of the overestimate. Several explana- 
tions may be offered for the variation in IOC 
year coefficients. Different inflation indices 
were used to normalize the data during dif- 
ferent time periods. The indices used may not 
have been appropriate. 

The IOC year variable used for time assumes 
a constant rate of change over the entire time 
period. It is possible that whatever factor the 
IOC year variable is attempting to measure 
was, itself, changing over time. Productivity 
changes in the work force are one possible ex- 
planation. Due to the magnitude of the error 
caused by the IOC year coefficient, it will be 
essential to identify the source of error before 
this model can be used for forecasting. Fu- 
ture work will focus on isolating the problem 
and developing solutions. 

Conclusions 

In order to accurately make any forecast us- 
ing mathematical or statistical modeling, 
several conditions must be meet. First, the 
structure of the model; i.e., the nature of the 
relationships, must be identified. Second, the 
parameters of the equation that are expected 
to vary, as input or outputs, need to be speci- 
fied. Third, those factors that remain con- 
stant must be identified and estimated. Fi- 
nally, the conditions under which the struc- 
tural equations and parameters remain sta- 
ble must be specified and tested. Only when 
thorough testing has indicated stability and 
accuracy over the expected range of forecast- 
ing requirements can a model be put to oper- 
ational use. 

The model identified in this paper is a fair 
predictor of general hardware development 
cost. As such, it proves that using many var- 
ied programs as a data base for estimating a 
cost model is a viable concept. The use of 
many data points from different technology 
domains has several advantages. First, it in- 
creases the number of degrees of freedom in 


the statistical analysis which allows more 
explanatory variables to be used. 

Second, the wider range of data available 
provides a deeper insight into the nature of 
the relationship between cost and various 
program factors. For example, a limited ana- 
lysis of spacecraft data may have led to the 
conclusion that quantity elasticities are al- 
ways greater than unity. In fact, production 
economies of scale should be achieved once 
the initial prototype stage is passed. 

Third, a model based on a wide range of tech- 
nologies should be more suitable for estimat- 
ing the cost of new designs that may have no 
historical analogies. Finally, validating the 
model over different time periods may im- 
prove the confidence in estimates made far 
into the future. The model described here 
demonstrates that such a model can be con- 
structed and will estimate cost within fairly 
reasonable bounds. 

In addition, several economic conclusions can 
be drawn from the data model. The analysis 
shows that significant economies of scale 
with respect to weight exist for nearly all 
types of development hardware. The more 
complex the hardware, the greater the econo- 
mies of scale. Also, the lower the weight of a 
subcategory, the greater the economies of 
scale are for that subcategory. Some classifi- 
cations, such as ships, even have disecono- 
mies of scale with respect to weight. The esti- 
mated elasticity of cost with respect to 
weight ranges from 0.43 to 1.30 with an aver- 
age values of approximately 0.65. Economies 
of scale with respect to unit quantities also 
are evident. The range of estimated elastic- 
ities is very wide, from 0.3 to 1.17 with the 
average around 0.58. Some types of systems 
have diseconomies of scale. These are mostly 
very low production quantity systems such 
as spacecraft. The conclusion is that a modi- 
fied learning curve such as Figure Id may be 
appropriate. 
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The use of a culture variable was proven ef- 
fective for combining different technologies 
in the same database. A methodology for de- 
riving a quantitative measure of culture was 
presented and shown to produce good results. 
For future space developments, culture may 
be the most significant variable the cost ana- 
lyst has to select. Weight and quantities will 
usually be given, but the particular hard- 
ware may not fall into any of the historical 
subcategories. It may also be possible to esti- 
mate culture for future programs using de- 
terministic methods, such as a function of the 
ratio between weight and quantity. Another 
possible method of estimating new cultures 
would be interpolation or extrapolation of ex- 
isting cultures. 

The inclusion of a time-based variable causes 
the effects of time to be removed from the 
other variables in the model. The model 
could be used for long range planning if the 
future effect of time could be predicted. It 
was found that the cost of programs is in- 
creasing with time, even after the effects of 
inflation are excised. The time-related cost 
growth is not at a constant rate. The magni- 
tude of cost growth appears to be from 0.0 to 
3.0 percent per year. The exact nature of this 
time-related phenomenon is not yet under- 
stood, although it is believed to be combina- 
tion of increasing performance, complexity 
and technology offset by improving produc- 
tivity and development methods. 


Finally, the benefits of design inheritance 
were clearly demonstrated. Substantial re- 
ductions in cost from using existing designs 
rather than starting from scratch are evident 
from the large negative coefficient of the gen- 
eration variable. Cost savings of about 22 
percent for each subsequent generation are 
predicted by the model. This fact has been 
used to great advantage on military acquisi- 
tion programs and should be incorporated 
whenever possible in the space program. 

The model does have some deficiencies. Most 
of the problems result from the wide range of 
estimated coefficients for subcategory models 
as shown in Table 3. The model of all data 
must effectively average these coefficients, 
which results in errors at the subcategory 
level. In addition, it was found that the mod- 
eling of time-related behavior (e.g., inflation, 
productivity, technology, etc.) is inaccurate. 
The model assumes that the rate change is 
constant but, in reality, it varies. 

The combination of these two deficiencies 
makes the specified model unsuitable for 
long-range estimates of advanced space pro- 
grams. Although the basic technique demon- 
strated here is sound, it must be refined even 
further to produce acceptable cost estimates. 
The specific weaknesses of the model have 
been identified and potential solutions will 
be implemented in the future. 
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But What Will It Cost? 

The History of NASA Cost Estimating 

by Joseph W. Hamaker 


Within two years of being chartered in 1958 
as an independent agency to conduct civilian 
pursuits in aeronautics and space, NASA ab- 
sorbed either wholly or partially the people, 
facilities and equipment of several existing 
organizations. These included the laborato- 
ries of the National Advisory Committee for 
Aeronautics (NACA) at Langley Research 
Center in Virginia, Ames Research Center in 
California, and Lewis Research Center in 
Ohio; the Army Ballistic Missile Agency 
(ABMA) at Redstone Arsenal Alabama, for 
which the team of Wernher von Braun 
worked; and the Department of Defense Ad- 
vanced Research Projects Agency (ARPA) 
and their ongoing work on big boosters. 1 

These were especially valuable resources to 
jump start the new agency in light of the 
shocking success of the Soviet space probe 
Sputnik in the autumn of the previous year 
and the corresponding pressure from an im- 
patient American public to produce some re- 
sponse. Along with these inheritances, there 
came some existing systems engineering and 
management practices, including project cost 
estimating methodologies. This paper will 
briefly trace the origins of those methods and 
how they evolved within the agency over the 
past three decades. 

The Origins of the Art 
World War II had caused a demand for mili- 
tary aircraft in numbers and in models that 
far exceeded anything the aircraft industry 
had even imagined before. While there had 
been some rudimentary work from time to 
time 2 to develop parametric techniques for 
predicting cost, there was certainly no wide- 
spread use of any kind of cost estimating be- 
yond a laborious build-up of work hours and 
materials. A type of statistical estimating 
had been suggested in 1936 by T. P. Wright 
in the Journal of Aeronautical Science . 3 


Wright provided equations which could be 
used to predict the cost of airplanes over long 
production runs, a theory which came to be 
called the learning curve. By the time the de- 
mand for airplanes had exploded in the early 
years of World War II, industrial engineers 
were happily using Wright’s learning curve 
to predict the unit cost of airplanes when 
thousands were to be built (and its still used 
today though the quantities involved are 
more likely to be hundreds instead of thou- 
sands). 

In the late 1940s the Department of Defense 
and especially the U.S. Air Force were study- 
ing multiple scenarios of how the country 
should proceed into the new age of jet air- 
craft, missiles and rockets. The Air Force 
saw a need for a stable, highly skilled cadre 
of analysts to help with the evaluation of 
these alternatives and established the Rand 
Corporation in Santa Monica, California, as 
a civilian think tank to which it could turn 
for independent analysis. Rand’s work repre- 
sents some of the earliest and most systemat- 
ic published studies of cost estimating in the 
airplane industry. 

Among the first assignments given to Rand 
were studies of first and second generation 
ICBMs, jet fighters and jet bombers. While 
the learning curve was still very useful for 
predicting the behavior of recurring cost, 
there were still no techniques other than de- 
tailed work-hour and material estimating for 
projecting what the first unit cost might be (a 
key input to the learning curve equation). 
Worse still, no quick methods were available 
for estimating the nonrecurring cost associ- 
ated with research, development, testing and 
evaluation (RDT&E). In the defense business 
in the early to mid-1950s, RDT&E had sud- 
denly become a much more important consid- 
eration for two reasons. First, a shrinking de- 
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fense budget (between World War II and the 
Korean War) had cut the number of produc- 
tion units of most Air Force programs. Sec- 
ond, the cost of new technology had greatly 
magnified the cost of development. The in- 
ability to nimbly estimate RDT&E and first 
unit production costs was a distinct problem. 

Fortunately, within Rand a cost analysis de- 
partment had been founded in 1950 4 under 
David Novick, who was drafted into the job 
because he was the only one around with any 
cost experience. This group at Rand proved to 
be prolific contributors to the art and science 
of cost analysis so much so that the literature 
of aerospace cost estimating of the 1950s and 
1960s is dominated by the scores of Rand cost 
studies that were published. 5 Novick and 
others at Rand deserve credit for developing 
and improving the most basic tool of the cost 
estimating discipline, the cost estimating re- 
lationship (CER), and merging the CER with 
the learning curve to form the foundation of 
aerospace estimating, which stands today. 6 

By 1951, Rand was devising CERs for air- 
craft cost as a function of such variables as 
speed, range, altitude, etc. Acceptable statis- 
tical correlations were observed at least ac- 
ceptable enough for the high-level compari- 
sons between alternatives that Rand 
was doing at the time. When the data was 
segregated by aircraft types (e.g., fighters, 
bombers, cargo aircraft), families of curves 
were discovered. Since each curve corre- 
sponded to different levels of complexity, the 
stratification helped clarify the development 
cost trends. Eventually, a usable set of pre- 
dictive equations was derived that was 
quickly put to use in Air Force future plan- 
ning activities. 

The use of the CERs and stratification were 
basic breakthroughs in cost estimating, espe- 
cially for RDT&E and first unit costs. For the 
first time, cost analysts saw the promise of 
being able to estimate relatively quickly and 
accurately the cost of proposed new systems. 


Rand extended the methods throughout the 
1950s and by the early 1960s the techniques 
were being acceptably applied to all phases of 
aerospace systems. 7 

The Early NASA Years 
In the spring of 1957 the Army Ballistic Mis- 
sile Arsenal (ABMA) in Huntsville, under 
the direction of Wernher von Braun, initiat- 
ed design studies on a large and advanced 
rocket booster that could be used for large 
DoD payloads then being conceptualized. 8 
Numerous design options were under consid- 
eration and all of the most promising needed 
cost projections. Von Braun’s team had long 
been flying experimental rockets, but pre- 
cious little cost data existed, and none exist- 
ed for the scale of the rockets that were com- 
ing off the drawing boards. Neverthe- 
less, estimates were being demanded. With 
the procedures that Rand had used on air- 
craft, data was pieced together and plotted 
against gross liftoff weight because this per- 
formance variable was known both for the 
historical data points and for the concepts be- 
ing estimated. The resulting CERs were at 
the total rocket level (engines being added 
separately based mainly on contractor esti- 
mates) and often did not inspire much confi- 
dence either by their correlation or their 
number of data points. 9 

Suddenly, in the fall of 1957 the Soviets 
launched Sputnik I and then, four weeks 
later, Sputnik II (carrying a dog), and the 
Army’s big booster work took on an entirely 
new importance. While vehicle configuration 
studies inspired by the Soviet success contin- 
ued at a rapid pace through 1958 and 1959, 
some momentous programmatic decisions 
were made regarding the ultimate manage- 
ment relationships between ABMA, the 
Army Redstone Project Arsenal (ARP A) and 
NASA. ABMA and von Braun, under ARPA 
sponsorship, were designing a massive rock- 
et called Saturn. The DoD, however, as 
ARPA’s parent organization, was coming to 
the conclusion that they did not need such a 
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super booster and was beginning to with- 
draw support over the objections of both 
ARPA and ABMA. In the end, by autumn of 
1959, both the Secretary of Defense and 
President Eisenhower had concluded that 
ABMA and the Saturn should be transferred 
to NASA. 10 In addition, a new home was 
found for the von Braun team by setting 
aside a complex within the borders of Red- 
stone Arsenal in Huntsville. 

By early fall of 1960, the Marshall Space 
Flight Center (MSFC) was operational. 

NASA’s first 10-year plan had been submit- 
ted to Congress in February 1960; it called 
for a broad program of Earth orbital satel- 
lites, lunar and planetary probes, larger 
launch vehicles and manned flights to Earth 
orbit and around the moon. The cost, esti- 
mated by analogies, intuition and guesses, 
was given as $1 billion to $1.5 billion per 
year. 11 

With the Kennedy Administration in office 
by early 1961, planning for a manned lunar 
landing project continued. President Kenne- 
dy and Vice President Johnson were both in- 
terested in options for moving ahead of the 
Soviets, and NASA was working on plans 
that could place an American on the lunar 
surface shortly after the turn of the decade. 

The orbiting of Yuri Gagarin in April 1961 
caused immediate questions from the Ad- 
ministration and Congress about the costs of 
accelerating the plans. Jim Webb, the NASA 
Administrator, had been briefed on $10 bil- 
lion cost estimates associated with the moon 
project. Prudently, he decided to give himself 
some rope and gave Congress a $20 to $40 
billion range. (The program was to cost about 
$20 billion ultimately.) 

Despite the magnitude of the cost projec- 
tions, in his State of the Union address in 
May 1961, President Kennedy established 
his famous goal of a lunar mission before the 


end of the decade. NASA was off and run- 
ning. MSFC took responsibility for the Sat- 
urn launch vehicles, and the new Manned 
Spacecraft Center (MSC) in Houston, created 
in mid- 1962 but operating before that out of 
Langley, was given responsibility for the 
payload — in this case the modules that would 
take the astronauts to the moon’s surface and 
back. 

While MSFC was being organized, the Jet 
Propulsion Laboratory (JPL) in California, 
in business as an Army research organiza- 
tion since the 1930s, was transferred to 
NASA from the Army. JPL had already built 
the Explorer satellite that had ridden an 
ABMA rocket into orbit as the country’s first 
successful response to Sputnik. JPL began its 
association with NASA by being assigned 
the lead center role for Agency planetary 
projects. As JPL began designing several 
planetary probes, including the Ranger se- 
ries of lunar spacecraft, the planetary series 
of Mariner spacecraft and the Lunar Survey- 
or spacecraft, they were dependent primarily 
upon contractor quotes for purchased hard- 
ware and their own work-hour and material 
estimates for inhouse work. 

As the pace of planning picked up, they be- 
gan to use an Air Force tool, the Space Plan- 
ner’s Guide, 12 a chapter of which is devoted to 
weight-based CERs for space project estimat- 
ing. In 1967, Bill Ruhland, a former Chrysler 
Saturn I-C manager, went to work at JPL 
and contracted with a new company called 
Planning Research Corporation (which had 
been started by some former analysts who 
had worked on the Space Planner’s Guide) to 
improve the CERs. 13 Ruhland stuck with es- 
timating, and went on to become NASA’s 
preeminent estimator for planetary space- 
craft throughout the 1970s and 1980s. PRC 
leveraged its early relationship with JPL 
and Ruhland by establishing cost modeling 
contracts with most of the other NASA cen- 
ters and dominating the development of 
NASA cost models for the next 25 years. 
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In March 1961, with launch vehicles, man- 
ned capsules and planetary spacecraft work 
underway, NASA dedicated the Goddard 
Space Flight Center (GSFC) as another de- 
velopment center. GSFC was assigned re- 
sponsibility for Earth orbital science satel- 
lites and soon had on the drawing board a 
number of spacecraft for which cost esti- 
mates were needed. The Orbiting Astronomi- 
cal Observatory, the Orbiting Geophysical 
Observatory and the Nimbus programs were 
all started early in the 1959-60 period and, 
like most other projects in the Agency at the 
time, experienced significant cost growth. 
GSFC organized a cost group to improve the 
estimates, first under Bill Mecca, and later 
managed by Paul Villone. In 1967 Werner 
Gruhl joined the office where he implement- 
ed numerous improvements to the GSFC 
methods. In later years he joined the Comp- 
troller’s office at NASA Headquarters as 
NASA’s chief estimator. 

Among the improvements creditable to 
GSFC during the late 1960s and early 1970s 
were: 1) spacecraft cost models that were sen- 
sitive to the number of complete and partial 
test units and the quality of the test units; 2) 
models devoted to estimating spacecraft in- 
struments; and 3) the expansion of the data- 
base through the practice of contracting with 
the prime contractor to document the cost in 
accordance with NASA standard parametric 
work breakdown structures (WBS) and ap- 
proaches. 14 

By 1965 most of NASA’s contractors were re- 
vising their traditional approach to cost esti- 
mating, which had relied upon the design en- 
gineers to estimate costs, replacing it with an 
approach that created a new job position that 
of trained parametric cost estimators whose 
job it was to obtain data from the design en- 
gineers and translate this information into 
cost estimates using established proce- 
dures. 15 At essentially the same time, cost es- 
timating was being elevated to a separate 
discipline within NASA Headquarters and at 


the NASA field Centers. This trend toward 
cost estimating as a specialization was 
caused by several factors. First, it was unre- 
alistic to expect that the design engineers 
had the interest, skills and resources neces- 
sary to put together good cost estimates. Sec- 
ond, during the preceding three years, the 
pace of the Gemini and Apollo programs had 
so accelerated that the Requests for Propos- 
als issued by the government typically gave 
the contractors only 30 days to respond — only 
parametricians had any hope of preparing a 
response in this short amount of time. Third, 
because of growing cost overrun problems, 
NASA cost reviews had increased notably 
and the reviewers were looking for costs with 
some basis in historical actuals — essentially 
a prescription for parametric cost estimating. 

At both MSC and MSFC, the cost estimating 
function was placed in an advanced mission 
planning organization. At MSC, it was em- 
bodied within Max Faget’s Engineering and 
Development Directorate, 16 and at MSFC it 
was within the Future Projects Office headed 
by Herman Koelle. 17 Faget, an incredibly 
gifted engineer, had already left his imprint 
on the Mercury, Gemini and Apollo pro- 
grams, and was a strong believer in an ad- 
vanced planning function with strong cost 
analysis. Koelle, a German engineer who, 
though not a member of the original team, 
had later joined von Braun, was also ex- 
tremely competent and very interested in 
cost. Koelle had, in fact, along with his depu- 
ty William G. Huber, assembled the very 
first NASA cost methodology in 1960, pub- 
lished first in an inhouse report 18 and then in 
1961 as a handbook that Koelle edited for 
budding space engineers. 19 

Out of the eye of the Apollo hurricane for the 
moment, both the MSFC and the MSC cost 
personnel now sought to regroup and at- 
tempt to make improvements in capability. 
In 1964 MSFC contracted with Lockheed and 
General Dynamics 20 to develop a more rigor- 
ous and sophisticated cost modeling capabil- 
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ity for launch vehicle life cycle cost modeling. 
This effort was led by Terry Sharpe of 
MSFC’s Future Projects Office. Sharpe, an 
Operations Research specialist interested in 
improving the rigor of the estimating pro- 
cess, led the MSFC estimating group as they 
managed the contractors’ development of the 
model and then brought it inhouse and in- 
stalled it on MSFC mainframe computers. 

Through about 1965 the only computational 
support in use by NASA estimators was the 
Freidan mechanical calculator. By the mid- 
1960s mainframe time was generally avail- 
able, and by the late 1960s the miracle of 
hand-held, four-function electronic calcula- 
tors could be had for $400 apiece— one per of- 
fice was the general rule. Throughout the 
early 1970s the hand-held calculator ruled 
supreme. By the middle 1970s, IMSAI 8080 
8-bit microcomputers made their appear- 
ance. Finally, by the late 1970s the age of the 
personal computer had dawned. Estimators, 
probably more than any other breed, imme- 
diately saw the genius of the Apple II, the 
IBM PC and the amazing spreadsheets: Visi- 
calc, Supercalc and Lotus 1-2-3. Civilization 
had begun. 

The resulting capability was extremely am- 
bitious for the time, taking into account a 
multitude of variables affecting launch vehi- 
cle life cycle cost. The model received signifi- 
cant notoriety, and once the CIA inquired if 
the MSFC estimators might make a series of 
runs on a set of Soviet launch vehicles. Busy 
with their own work, the estimators de- 
murred. The CIA pressed the case to a higher 
level manager, a retired Air Force colonel. 
Suddenly the MSFC estimators discovered 
that they had been mistaken about priori- 
ties. The runs were made and the CIA ana- 
lysts went away happy. 

Later in 1964 after a reorganization, man- 
agement of the MSFC cost office was taken 
over by Bill Rutledge who went on to lead the 
MSFC cost group for more than 20 years. 


Rutledge steadily built the MSFC cost 
group’s strength until it was generally recog- 
nized in the late 1960s as the strongest cost 
organization within the Agency. One of Rut- 
ledge’s more outstanding innovations was 
the acquisition of a contractor to expand and 
maintain an Agencywide cost database and 
develop new models. The REDSTAR (Re- 
source Data Storage and Retrieval) database 
was begun in 1971 and is still operational to- 
day, supporting Agencywide cost activities. 
The contract was originally awarded to PRC 
and, under Rutledge’s management, devel- 
oped numerous models throughout the 1970s 
and 1980s. 

MSFC also established a grassroots cost esti- 
mating organization within the MSFC Sci- 
ence and Engineering laboratories. This 
group was managed by Rod Stewart for a 
number of years. After his retirement from 
NASA, Stewart, along with his wife Annie, 
authored an outstanding series of cost esti- 
mating books. 21 In 1966, MSC, working in 
parallel to the MSFC activities, contracted 
with General Dynamics 22 and Rand 23 to im- 
prove their spacecraft estimating capability. 
The MSC cost group also significantly im- 
proved their capabilities during this period 
under the able management of Humboldt 
Mandell, who was later to play a leading role 
in the Shuttle, Space Station and Space Ex- 
ploration Initiative cost estimating activi- 
ties. 

By 1967 both the MSC and MSFC cost esti- 
mating organizations were beginning to ob- 
tain the first historical data from the flight 
hardware of the Apollo program. This includ- 
ed cost data on the Saturn IB and Saturn V 
launch vehicles by stage, and on the Com- 
mand and Service Module (CSM) and the Lu- 
nar Excursion Module (LEM) at the major 
subsystem level. Fairly shallow data by to- 
day’s standards, it was considered somewhat 
of a windfall to the NASA estimators who 
had been struggling along with two- and 
three-data point CERs at the total system 
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level. The Project Offices at MSC and MSFC 
compiled the data between 1967 and 1969 
and documented the results in the unpub- 
lished Apollo Cost Study (preserved today in 
the JSC and MSFC cost group databases). 
Eventually this was supplemented by paying 
the CSM prime contractor to retroactively 
compile the data in a WBS format useful for 
parametric cost estimating. 24 Despite these 
improvements, one Rand report in 1967 la- 
ments that the number of data points for cost 
estimating was depressingly low. . . “only one 
subsystem contains more than four data 
points and this paucity of data precludes the 
application of statistical techniques either in 
the development of the CERs themselves, or 
in the establishment of confidence levels for 
the predictive values generated by the 
CERs.” 25 

While most of the science programs were 
managed out of JPL and GSFC, the research 
centers (Ames, Langley, and Lewis) were 
also given development projects from time to 
time. Ames managed the Pioneer planetary 
probes, Langley managed the Lunar Orbiter 
and the Viking Mars mission, and LeRC 
managed the Centaur project. Generally, the 
costs were estimated using models from the 
other Centers. 

The Shuttle Era: Promise of Low Cost 
By 1968 the nation was immersed in social 
and political turmoil, the Vietnam War and 
the attempt to build the Great Society. 
Though the accomplishment of the first man- 
ned lunar landing was not to occur until the 
following year, the budget that NASA re- 
ceived was lower than the previous year and 
broke the trend of ever increasing flows of 
money that the Agency had enjoyed since its 
creation a decade before. NASA realized that 
the dream of building directly on the expend- 
able Saturn launch vehicle technology, 
building Earth orbital and lunar orbital 
space stations, continuing exploration of the 
lunar surface and mounting an expedition to 
Mars were not in the immediate plans. 


By early 1969, while the ongoing Apollo pro- 
gram prepared for the Apollo 11 mission to 
the moon on which humans would land for 
the first time, future planning activities 
within NASA had been scaled back from the 
overly ambitious, broad set of space activities 
to focus on the crucial next step. Space sta- 
tions, moon bases and Mars missions all 
needed low-cost, routine transportation from 
the Earth’s surface to low Earth orbit. If the 
budget realities precluded doing everything 
at once, then the next thrust would be in low 
Earth orbit transportation as a first building 
block to all the rest. 

A task force was assigned in March 1969 to 
study the problem and recommend options 
for further study. 26 This report called for the 
development of a new space shuttle system 
that could meet certain performance and 
cost-per-flight objectives. Many options were 
examined, but the fully reusable two-stage 
was the preferred choice because it seemed to 
offer the lowest recurring cost. Concurrently 
with these inhouse assessments, four paral- 
lel Phase A (i.e., conceptual design) studies 
had been awarded to General Dynamics, 
Lockheed, McDonnell Douglas and North 
American (today’s Rockwell International). 
For most of 1969 these studies proceeded 
apace, churning out massive stacks of paper 
designs, along with cost numbers that gave 
the impression that all was well. For around 
$10 billion in development costs, the most re- 
usable Shuttle configurations offered recur- 
ring costs of only a few million dollars per 
flight. 

As the Phase A studies neared completion in 
late 1969, however, two cost-related prob- 
lems began to emerge. First, NASA’s com- 
munications with the Office of Management 
and Budget (OMB) revealed that the outlook 
for the NASA budget was not good. The pro- 
jections showed that continued reductions in 
NASA’s funding were inevitable; the lower 
budget numbers did not match the amount 
needed to fund the favored Shuttle designs. 
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Second, as NASA reviewed the contractors 
cost estimates for the Shuttle and compared 
the numbers to their own estimates, it be- 
came clear that no one in the industry or the 
government had a good handle on what the 
Shuttle could be expected to cost. 27 

The problem with the estimates was analo- 
gous data. A winged, reusable spaceship had 
never been built before and all the cost esti- 
mates were being based on extrapolations 
from large aircraft such as the C-5, B-52, 
B-70 (for wings, fuselage, landing gear, etc.), 
from the Saturn (for tanks, thrust structure, 
etc.) and from the Apollo capsules (for crew 
systems). The problem was compounded by 
the scope of the estimating job. All the var- 
ious designs being contemplated overloaded 
the estimating resources that NASA had at 
the time. The entire complement of NASA 
estimators at the two lead Centers (JSC and 
MSFC) numbered only eight people, yet cost 
was to be one of the most key variables in the 
decision making process concerning the 
Shuttle. 28 

Because the magnitude of the upfront costs of 
the fully reusable systems had not yet been 
adequately estimated, NASA proceeded into 
Phase B in mid- 1970 with the intent of put- 
ting more meat on the bones of the skeletal 
designs. Meanwhile, negotiations with the 
Office of Management and Budget continued 
concerning the budget outlook, and the num- 
bers got lower and lower. Slowly, the cost es- 
timates became more realistic just as the 
Phase B studies were nearing completion in 
the summer of 1971. 

The studies were extended so that cost cut- 
ting measures could be investigated. First, 
expendable drop tanks were substituted for 
reusable interior tanks. Then the flyback 
booster was scrapped, first for expendable 
liquid rocket boosters, then for expendable 
solid rocket boosters. Taken together, these 
reductions made it possible to barely fit the 
Shuttle’s development within the OMB 


guidelines, but each change had added to the 
recurring cost per flight. 29 

But the Shuttle peak year funding versus the 
OMB budget cap was not the only cost ques- 
tion dogging the Shuttle. For the mandated 
Mercury, Gemini and Apollo programs, mon- 
ey had flowed without any requirement for 
the Agency to show economic justification for 
the projects. When the idea of a Shuttle sys- 
tem was floated in 1969 as part of NASA’s 
plans after Apollo, the OMB decided that 
such an expensive undertaking ought to 
show some economic benefits that out- 
weighed the costs. Because the analytical 
skills for an economic justification did not ex- 
ist inhouse and NASA thought it wise to 
have independent support for the Shuttle, 
the Agency hired the Aerospace Corporation, 
Lockheed and economist Oskar Morgenstern 
and his company Mathematica to develop the 
data OMB wanted to see. Morgenstern 
turned the economic analysis over to a young 
protege named Klaus Heiss. Heiss put to- 
gether an impressive study 30 that compared 
the life cycle costs of the Shuttle with the 
costs of the equally capable expendable 
launch vehicles. 

One of the more important arguments for the 
Shuttle case was that payloads on the Shut- 
tle would cost considerably less than pay- 
loads on expendables, a notion that was 
based on an extensive cost estimating study 
done for NASA by Lockheed. 31 This study, a 
classic for its scope, originality and method- 
ology, nevertheless reached an exactly wrong 
conclusion. 

It is known now that Shuttle payloads actu- 
ally cost more than those that fly on expend- 
able launch vehicles due to the strenuous 
safety review process for a manned vehicle. 
But Lockheed forecasted that the payload de- 
velopers would save about 40 percent of their 
costs from the advantages offered by the 
Shuttle. The advantages were thought to be 
that: 1) the relatively high weight lifting per- 


31 




READINGS IN PROGRAM CONTROL 


formance and payload bay volume offered by 
the Shuttle would allow payloads to ease up 
on lightweighting and miniaturization, 
which are cost drivers; 2) the Shuttle would 
allow retrieval and refurbishment of satel- 
lites instead of buying additional copies as 
was necessary with expendable rockets; and 
3) a single national launch system such as 
the Shuttle would allow standardization of 
payloads instead of multiple designs config- 
ured for the plethora of expendable vehicle 
interfaces. Finally, it was Aerospace’s job to 
determine the payload requirements and 
produce traffic models, and they ultimately 
forecasted the need for 60 Shuttle flights per 
year. 32 While the Shuttle payload benefits 
and flight rates were both flawed assump- 
tions, Klaus Heiss constructed a discounted 
cost benefit analysis that asserted savings in 
the billions. At the least, the Aerospace, 
Lockheed, Mathematica work sent the OMB 
accountants to murmuring. 

President Nixon finally gave the nod, and 
the Shuttle’s detailed design began in the 
summer of 1972 under contract to the win- 
ning prime contractor, North American — 
though this did not end the debate over the 
worthiness of the project. 33 All through 1973 
NASA was very involved in extensive cap- 
ture/cost analyses to produce data to answer 
Congressional, GAO and OMB inquiries 
about the Shuttle’s economic forecasts. These 
analyses were NASA inhouse extensions of 
the work done by Mathematica, Lockheed 
and Aerospace. The studies consumed most 
of the resources of the MSFC and JSC cost 
groups as well as Headquarters program of- 
fice personnel. They compared the discount- 
ed life cycle costs of capturing the NASA and 
DoD payloads with the Shuttle versus ex- 
pendable launch vehicles. The Shuttle case 
was finally determined to yield a 14 percent 
internal rate of return and $14 billion of 
benefits (in 1972 dollars). This data was used 
as the final reinforcement of the Shuttle pro- 
gram commitment. 


Declining Budgets, Rising Costs 
Once Shuttle development was safely under- 
way by 1974, most of the estimating talent of 
the Agency was turned to various kinds of 
scientific satellite estimating. As NASA’s 
budget declined in the 1970s, both JPL and 
GSFC pioneered such economies as the use of 
the protoflight concept in spacecraft develop- 
ment. Before the 1970s NASA had proto- 
typed most spacecraft (i.e., built one or more 
prototypes which served as ground test arti- 
cles) before building the flight article. In the 
protoflight approach, only one complete 
spacecraft is built, which serves first as the 
ground test article and is then refurbished as 
the flight article. The protoflight approach 
theoretically saves money. However, these 
savings must be balanced against the cost of 
refurbishing the test article into a state 
ready for flight, the cost of maintaining more 
rigid configuration control of the ground test 
article to insure its eventual flight worthi- 
ness, and the increased risk of having less 
hardware. 

Other attempts were made to lower cost 
without much success. Low estimates based 
on wishful thinking concerning off-the-shelf 
hardware and reduced complexity proved un- 
realistic, and overruns began to breed more 
overruns as projects underway ate up the 
funds other projects had expected. 

Meanwhile, as NASA Headquarters contin- 
ued to guide the overall programs, handle 
the political interfaces, foster other external 
relations, and integrate and defend the 
Agency budget, a need was seen to strength- 
en the Washington cost analysis function. 34 
Having moved to the Headquarters Comp- 
troller’s Office from GSFC in 1970, Werner 
Gruhl set up an independent review capabil- 
ity under Mai Peterson, an assistant to the 
Comptroller. Gruhl aggressively championed 
the constant improvement of the database. 
Gruhl and Peterson’s greatest contribution 
was probably their relentless urging for real- 
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istic estimates. They also initiated an annual 
symposium for all NASA estimators and 
were instrumental in helping to establish a 
process for Non-Advocate Reviews (NARs) 
for potential new projects. 

The NAR was instituted as a required miles- 
tone in which each major new project had to 
prove its maturity to an impartial panel of 
technical, management and cost experts be- 
fore going forward. As part of the NAR pro- 
cess, Peterson and Gruhl, working with a rel- 
atively small staff of one to three analysts, 
undertook to perform independent estimates 
of most of the major new candidates for au- 
thorization. Peterson largely devoted himself 
to penetrating reviews of the technical and 
programmatic readiness, the underpinning 
of the cost estimate. Gruhl, using mostly 
models of his own developed from the RED- 
STAR database, generated his own esti- 
mates. Together they were a formidable 
team and undoubtedly reduced the cost over- 
run problem from what it would have been 
without the NAR. 

Another significant milestone in cost esti- 
mating that occurred during the 1970s was 
the emergence of the Price Model. First de- 
veloped within RCA by Frank Freiman, the 
model began to be marketed in 1975 by RCA 
as a commercially available model. Frei- 
man’s brainchild was arguably the single 
most innovative occurrence in parametric 
cost estimating ever. His genius was to see 
hardware development and production costs 
as a process governed by logical interrela- 
tionships between a handful of key variables. 
Probably feeling his way with intuition and 
engineering experience more than hard data, 
Freiman derived a set of algorithms that 
modeled these relationships. The resulting 
model could then be calibrated to a particu- 
lar organization’s historical track record by 
essentially running the model backward to 
discover what settings for the variables gave 
the known cost. Once calibrated, the model 
could be run forward using a rich set of tech- 


nical and programmatic factors to predict the 
cost of future projects. While the Price 
models are applicable to a wide range of in- 
dustries in addition to aerospace, the model 
first found use in the aerospace industry. 
NASA encouraged Freiman to market his in- 
vention, and actually provided him with data 
for calibrating the model after observing its 
potential in Shuttle cost estimating. 35 The 
success of the Price model inspired the devel- 
opment of several other commercial cost 
models with application to hardware, soft- 
ware and the life cycle. 

By the late 1970s and into the mid-1980s, the 
cost of NASA projects was a serious problem. 
It was now obvious that Shuttle payloads 
cost more, not less, than payloads on un- 
manned vehicles. Overruns were worse than 
ever despite better databases, better models, 
better estimators, and more stringent Head- 
quarters reviews. It seemed that NASA was 
in danger of pricing itself right out of busi- 
ness. 36 At JSC, Hum Mandell, assisted by 
Richard Whitlock and Kelley Cyr, initiated 
analyses of this problem. Making imagina- 
tive use of the Price model, 37 they found that 
NASA’s culture drives cost and that the com- 
plexity of NASA projects had been steadily 
increasing, an idea also advanced by Gruhl. 
Mandell argued persuasively to NASA man- 
agement for a change in culture from the ex- 
otically expensive to the affordable. At the 
same time, he argued that estimates of fu- 
ture projects needed to account for the stead- 
ily increasing complexity of NASA projects. 

Recent Years 

Once the Space Shuttle had begun oper- 
ations, NASA turned its attention once again 
to defining a Space Station. After Pre-Phase 
A and Phase A studies had analyzed several 
configurations, in 1983 NASA ran a Wash- 
ington-based, multi-center team called the 
Configuration Development Group (CDG) to 
lead the Phase B studies. The CDG was led 
by Luther Powell, an experienced MSFC pro- 
ject manager. For his chief estimator, Powell 
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chose O’Keefe Sullivan, a senior estimator 
from the MSFC cost group. Sullivan had just 
completed managing the development of the 
PRC Space Station Cost Model, 38 an innova- 
tive model that created a Space Station WBS 
by cleverly combining historical data points 
from parts of the Shuttle Orbiter, Apollo 
modules, unmanned spacecraft and other 
projects. This model was distributed and 
used by all four of the Work Package Centers 
and was probably the most satisfactory para- 
metric cost model ever developed by NASA. 
Work Package 1 (WP-1) was at MSFC, with 
responsibility for the Station modules; WP-2 
was at JSC with responsibility for truss 
structures, RCS and C&DH; WP-3 was at 
LeRC with responsibility for power; and 
WP-4 was at GSFC with responsibility for 
platforms. Sullivan used the model to esti- 
mate the project at between $11.8 and $14 
billion (in 1984 dollars). The content of this 
estimate included the initial capability, 
eight-person, 75-kilowatt station and space 
platforms at two different orbital locations, 
with additional dollars required later to grow 
the program to full capability. 39 

Meanwhile, NASA Administrator Jim Beggs 
had been negotiating with the OMB for sup- 
port to start the project. Under pressure to 
propose something affordable, Beggs com- 
mitted to Congress in September 1983 that a 
Station could be constructed for $8 billion, a 
rather random number in light of the known 
estimates and the fact that the conceptual 
design had never settled down to an extent 
necessary for a solid definition and cost esti- 
mate. Nevertheless, the Agency pushed 
ahead with the Phase B studies and by fall 
1987, needing to narrow the options in con- 
figurations still being debated between the 
Centers, established a group called the Criti- 
cal Evaluation Task Force (CETF), quar- 
tered at LaRC and led by LaRC manager Ray 
Hook. Hook brought Bill Rutledge in from 
MSFC to lead the cost analysis effort, and 
Rutledge assembled a team made up of esti- 
mators representing the Work Package Cen- 


ters and Headquarters (Bill Hicks, Richard 
Whitlock, Tom LaCroix, and Dave Bates). 
Over a period of a few intense weeks, they 
generated the cost of the new baseline, 
which, even after significant requirements 
had been cut, still totaled at least $14 billion. 

NASA reluctantly took this cost to the OMB. 
Seeking to inspire a can-do attitude among 
the CETF team, NASA management passed 
out buttons containing the slogan We Can Do 
It! One senior estimator, who had seen it all 
before, modified his button to read We Can 
Do It For $20 Billion! 40 Amid great political 
turmoil, the Space Station was finally given 
a go-ahead. Despite contractor proposed costs 
that were more unrealistically optimistic 
than usual, the source evaluations were com- 
pleted and contracts were awarded for the 
four work packages. The project managed to 
survive several close calls in the FY1988 
through FY1991 budgets, though with stead- 
ily escalating costs and several iterations of 
requirements cutbacks and redesigns. Like 
the purchase of a car, the sticker price in- 
cludes nonrecurring cost only, and this is the 
cost NASA had always quoted Congress for 
new projects, including the Space Station. 
During the long and winding road of gaining 
Congressional authority for the Station, 
NASA was asked to include other costs such 
as Station growth, Shuttle launch costs, op- 
erations costs, and various other costs, which 
led to confusion and charges of even more 
cost growth than actually occurred. 

As this is being written, NASA is actively de- 
signing and estimating the cost of several 
major future programs including the Earth 
Observation System, the National Launch 
System and the Space Exploration Initiative, 
among others. Each of these programs, like 
most NASA programs before them, is unique 
unto itself and presents a new set of cost esti- 
mating challenges. At the same time, the re- 
cent years of growth in budget resources that 
NASA has enjoyed seems to have run its 
course. In an era of relatively level budget 
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authority, NASA is seeking ways to maxi- 
mize the amount of program obtainable. New 
ideas on this topic abound. Total Quality 
Management, Design to Cost, Concurrent 
Engineering and a number of other cultural 
changes are being suggested as a solution to 
the problems of high cost. As usual, the 
NASA estimating community is in the mid- 
dle. Armed with data from the past, which 
somehow must be adapted to estimate the fu- 
ture, they attempt to answer the all impor- 
tant question: But what will it cost? 

So brief a treatment of the history of NASA 
cost estimating leaves so much unsaid that 
apologies are in order. Nothing was men- 
tioned of the aeronautical side of NASA, yet 
they estimate the cost of projects that are no 
less important to the nation than the space 


projects focused upon here. The Kennedy 
Space Center facilities and operations cost- 
ing was not mentioned, though nothing 
NASA has sent to space could have been sent 
without them. Whole projects from which 
much was learned about cost estimating (Vi- 
king, Skylab, Spacelab, Centaur-G, Hubble 
Space Telescope, Galileo, Magellan, Ulysses 
and many others) had to be left unexplored. 
Even when touched upon, many subjects 
were given only the barest of treatments, the 
expansion left for other studies. 

Finally, while this paper unfairly singles out 
a dozen or so individuals, another few score 
men and women who have labored hard in 
the crucial and controversial business of 
NASA cost estimating will not see their 
names here. They are saluted anyway. 
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Improving Cost Efficiency in Large Programs 


by John D. Hodge 

This paper examines the question of cost, 
from the birth of a program to its conclusion, 
particularly from the point of view of large 
multi-center programs, and suggests how to 
avoid some of the traps and pitfalls. Empha- 
sis is given to cost in the systems engineering 
process, but there is an inevitable overlap 
with program management. (The terms sys- 
tems engineering and program management 
have never been clearly defined.) In these 
days of vast Federal budget deficits and in- 
creasing overseas competition, it is impera- 
tive that we get more for each research and 
development dollar. This is the only way we 
will retain our leadership in high technology 
and, in the long run, our way of life. 

One of the most vexing aspects of managing 
large programs within NASA (or any other 
high technology government programs) is 
how to allocate program funds in a way that 
is best for the program. One of the major rea- 
sons is that the role of cost changes through- 
out the phases of the program. Another rea- 
son is that total cost is not all that easy to de- 
fine; yet another is that funding, which is 
based on annual appropriations, is almost 
never consistent with fiscally efficient pro- 
gram spending rates. The net result is that 
program costs almost always escalate and 
inordinate time is spent controlling costs at 
the expense of maintaining performance or 
schedule. 

Many studies have tried to address this prob- 
lem. They show that program costs will esca- 
late by at least a factor of three, from approv- 
al to completion. The studies suggest a num- 
ber of guidelines that should be followed if 
costs are to be kept down, including clear 
definition of requirements, stable manage- 
ment and strong central control. Unfortu- 
nately, these factors are not always under 
the control of the program manager. 


The principles are simple. First, define very 
carefully what it is you are trying to do. 
Check everything you do against that base- 
line, even if it has to be changed, and resist 
change once the decisions have been made. 
Second, break up the program into manage- 
ably sized deliverables that can be measured 
in terms of cost, schedule and performance, 
and define the interfaces between them. 
Third, continuously assess the risks to suc- 
cess as the program proceeds, and modify 
only as necessary. 

Requirements Traceability 
Most studies have shown that the primary 
reason for cost escalation is that not enough 
time or resources are spent in defining the 
program. It is clear that you cannot control 
what you have not or cannot define. It is dur- 
ing this period that some of the most elegant 
systems engineering should be performed, 
especially in understanding the cost of every 
requirement and its systems implication. 
Even if the definition is adequate during the 
early phases of the program, it is imperative 
that great vigilance be exercised in main- 
taining the baseline definition of the pro- 
gram and the fundamental reasons for doing 
the program. 

This process establishes a small but influen- 
tial part of the program office, preferably 
within the systems engineering organiza- 
tion. The program office must be dedicated to 
the traceability of requirements and to en- 
suring that a clear path exists from program 
rationale to program requirements to sys- 
tems requirements to systems design. Too of- 
ten, once a design has been established, 
changes are proposed and enacted that bear 
little relationship to the original premises of 
the program. As will be discussed later in 
this paper, there are many reasons for 
change, but where possible, changes should 
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be considered during the formulation of the 
program and not later when the program 
structure is in place and the program is in 
progress. Change is almost always costly; re- 
quirements traceability provides a bulwark 
against which the program manager and the 
systems engineer can stand and defend. 

Baseline Cost, Schedule 
and Performance 

The three main parameters in the control 
process — cost, schedule and performance — 
are the program manager’s bread and but- 
ter. Again, program definition is vital and 
necessary from the very beginning. It may 
be argued that clear definition is not possi- 
ble, particularly early in the program; nev- 
ertheless, an approved, traceable baseline, 
although it may alter, must be known at any 
given time, and must include everything in 
the program. The “I forgots” can kill you. 
The key to success in handling these three 
parameters is to manage the balancing act 
between them. Cost, schedule and perfor- 
mance are usually dependent variables and 
at various times, one or another may assume 
greater or lesser importance. A single vari- 
able, however, should never be changed 
without knowing the impact on the other 
two. Within the NASA culture, performance 
is generally the predominant factor, and 
schedule is a distant second. Cost tends to be 
considered mostly in the context of the annu- 
al appropriation, but from the point of view 
of the program manager, all three param- 
eters must be defined and approved continu- 
ously, which is a function of the systems en- 
gineering process. 

Program Risk Analysis 
In recent years, especially since the Chal- 
lenger accident, program risk analysis has 
come to be used largely in the context of crew 
safety, but this is only a part of program 
risk. Basically, program risk analysis asses- 
ses the probability of meeting requirements 
as changes occur. A number of analytical 
tools now available can be used to under- 
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stand the relationships between cost, perfor- 
mance and schedule. Again, a small group 
within the systems engineering organization 
should be dedicated to understanding the 
impact of any change on all three param- 
eters. Armed with this information, risk can 
be reduced in many ways. Adding more mon- 
ey, reducing the performance requirements, 
or extending the schedule are most often 
used. A competent systems engineer will 
know the relationships between these three 
variables and the impact of any situation on 
the total program. 

The Role of Cost in Phased Procurement 
The most common form of procuring high 
technology capability within the Federal 
Government is known as phased procure- 
ment. The theory behind this procurement 
method is that commitment to the program 
gradually increases with time and in dis- 
crete stages. Within NASA, there are four 
standard phases; others are beginning to 
creep in as the ability to establish new pro- 
grams becomes more difficult and the dura- 
tion and cost of operations becomes a more 
significant part of total program costs. The 
role of cost is different in each of the phases. 
The phases are: 

Pre-Phase A: This is a very unstructured 
period that examines new ideas, usually 
without central control and mostly ori- 
ented toward small studies. This period 
can last for a decade or more and produces 
the list of ideas and alternatives from 
which new programs are selected. 

Phase A: Sometimes called the feasibility 
phase, this is a structured version of the 
previous phase. Usually a task force or 
program office is established, and multi- 
ple contracts will be awarded. The goal of 
this phase, which may last for several 
years but usually is limited to one or two 
years, is to decide whether a new pro- 
gram will be started and what its purpose 
and content should be. This phase repre- 
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sents less than one percent of the total 
program costs. Nevertheless, it is largely 
a systems engineering effort and sets the 
stage for everything that follows. 

Phase B: Sometimes known as program 
definition, this phase is the most impor- 
tant in establishing the basic parameters 
of the program. By the time this phase is 
finished (a period of two or three years), 
the program rationale, cost, schedule, 
performance, management style and the 
most likely technical solution will have 
been established. This phase usually in- 
volves multiple contracts to establish a 
variety of ideas and a competitive envi- 
ronment, should the program proceed. 
Cost is continuously assessed as a func- 
tion of design solutions relative to basic 
requirements. Studies indicate that from 
five to ten percent of the total program 
costs will need to be expended if control is 
to be maintained over the program dur- 
ing Phases C and D. 

Phase C/D: Originally separate phases, 
this period covers design, development, 
test and evaluation. Contracts may be 
open to all qualified bidders or only to 
those involved in the previous phase. Al- 
though competition is not usually open 
between Phases C and D, commitment to 
Phase D depends on a successful and ac- 
ceptable design. In past programs, two- 
thirds of the total program cost was ex- 
pended during this period. The systems 
engineering role has begun to shift to- 
ward systems specification and systems 
interfaces. The secret to cost control is a 
sound definition of end items and their in- 
terfaces with a tight hold on changes. 

Phase E: In most past programs, the oper- 
ations costs were less than 20 percent of 
the total cost. This was because there was 
a definite end to a relatively short-term 
program. In recent years, particularly in 
the manned programs, the length of the 


operational phase has increased signifi- 
cantly. In the case of the Shuttle, it could 
be conceived as indefinitely long. For this 
reason, life cycle costs should be a major 
consideration from the beginning. 

Selling the Program 

The definition of a new start within NASA 
varies by program and organization but can 
generally be said to occur at the beginning of 
Phase B. Prior to that time, the program 
manager is selling the program. The total 
expenditure of funds during the selling peri- 
od is usually far less than one percent of the 
final program costs; this is, however, when 
the basic parameters of the program are es- 
tablished. It is a time of building constitu- 
ents both inside and outside the Agency. As- 
suming that a feasible technical solution is 
available and an acceptable management 
scheme can be provided, much of the debate 
about whether a program should be ap- 
proved centers largely around the question 
of cost. Of course, with only preliminary de- 
signs available, only cost estimates can be 
made and these are obtained from standard 
cost models. 

Cost Estimating 

During Phase A of the program, when the 
most rudimentary designs are available, it is 
essential that program cost estimates are 
made before the program start can be autho- 
rized. Estimates are made using cost models 
that have been developed on the basis of past 
experience on similar programs. These 
models are among the most arcane devices 
invented by engineers, so a few words on 
how they work are appropriate. 

Past experience is captured by documenting 
the cost of each system on the basis of 
weight. Regression analysis is performed to 
determine a straight line log relationship. 
Once the weight of the system has been esti- 
mated, the cost can be determined. This esti- 
mate is multiplied by a complexity factor to 
allow for the risk associated with the select- 
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ed technology and may vary from as little as 
0.50 to 2 or more. This is repeated for each 
system, and the total becomes the baseline 
cost. This total is multiplied by a factor to al- 
low for systems engineering and testing by 
the prime contractor. This is known as the 
“prime wrap” factor and is again determined 
based on all relevant past experience. 

All prime contractor estimates are added 
and then multiplied by a second factor 
known as the “nonprime wrap.” This is the 
cost of government work. Finally, a reserve 
factor is used to allow for problems during 
the program. There are separate cost models 
for manned and unmanned programs, which 
are significantly different. In general, for the 
unmanned programs, about 40 cents of every 
dollar goes to hardware, and in the manned 
programs, about 20 cents. 

These cost models pose a great many prob- 
lems. First, they are normalized on the basis 
of weight. Clearly this is not valid in all 
cases, particularly structure. Second, they do 
not explain why the costs are what they are. 
Factors such as management style, procure- 
ment strategy and test philosophy are not 
differentiated. Third, they include all past 
experience, including errors and overruns. 
In this respect, these cost models assume no 
learning curve. As it was in the beginning, is 
now, and forever shall be! They must there- 
fore be used with great caution. From the 
systems engineer’s point of view, these cost 
models can be used to assess the relative 
costs of various design solutions; on an abso- 
lute basis, however, they are of little use. 

So far we have been able to make a tentative 
estimate of the cost of the flight system. To 
this must be added the cost of new facilities, 
including launch, test beds, flight oper- 
ations, networks and data reduction, among 
others, and finally the cost of operations. It is 
at this point that the program manager faces 


the first dilemma: What should be included 
in the program cost? That sounds like a sim- 
ple question, but it is complicated by the fact 
that not all costs are under the control of the 
program manager, nor is he or she responsi- 
ble for justifying all of the associated appro- 
priations. 

For example, launch costs are provided by 
the Office of Space Flight, network costs are 
provided by the Office of Operations, and civ- 
il service costs are provided by the research 
and program management fund managed by 
the Office of the Comptroller. New buildings 
are provided under the construction of facili- 
ties budget. In addition, most new program 
managers are surprised to find that a tax 
based on the number of civil servants work- 
ing on the program varies from Center to 
Center, and neither the number of people 
nor the level of tax is under the control of the 
program manager. Taxation without repre- 
sentation! Despite this dilemma, the systems 
engineer should include all of these factors 
in the cost estimate because the chosen de- 
sign will affect all of them; overall program 
costs are as important to the Agency as di- 
rect program costs. 

Program costs tend to be presented as only 
those costs that are under the control of the 
program manager. No matter how much this 
limitation is stated in presentations, it is as- 
sumed that it is the total program cost (espe- 
cially when it is a popular program) that has 
the support of the Executive branch, the 
Congress and other constituencies. It is no 
wonder that the average program increases 
in cost by a factor of about three from the 
time of approval to completion and that most 
program managers during this period are ac- 
cused of everything from naivete to self- 
deception to outright lies. There is the added 
ethical question that if all costs were actual- 
ly presented, the program would not be ap- 
proved! 
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Defining the Program 

This phase of the program, usually known as 
Phase B, will take from one to two years. The 
purpose is to take the various concepts con- 
sidered in Phase A and select a single valid 
solution. By the time Phase B is over, a clear 
set of requirements should be available with 
a complete set of functional specifications 
and a cost estimate based on preliminary de- 
sign concepts rather than on cost models. 
These are primarily produced by the systems 
engineering organization and include at 
least one preliminary design and selected 
technologies with well-understood risks as- 
sociated with those technologies. 

Don Hearth, former director of the NASA 
Langley Research Center, performed a study 
on how much this phase has cost for various 
past programs as compared to the success of 
the program in later phases. Success was 
measured as the ability to maintain perfor- 
mance, schedule and cost as determined at 
the end of Phase B. He concluded that the 
most successful programs spent between five 
and ten percent of the total program cost in 
Phase B. The scope was limited to unmanned 
programs, but the rationale can reasonably 
be extended to manned programs. 

Apart from establishing a credible function- 
al system specification, it is essential to de- 
termine the management structure, the pro- 
curement strategy and a baseline cost for the 
life of the program, including the cost of op- 
erations. Once again, the primary method 
for cost estimating is the cost model, but 
there should be sufficient detail available to 
check the model with bottom-up costs based 
on feasible design solutions. The systems en- 
gineer is responsible for comparing these 
two cost estimating techniques. It is unwise 
to proceed to the next phases unless some 
bottom-up cost estimating has been per- 
formed. 

Perhaps the most important product of this 
phase is a complete work breakdown struc- 


ture. Again, this is largely the responsibility 
of the systems engineering organization. 
The axiom to be followed is, “You cannot 
control what you have not defined.” 

Work Breakdown Structure 
Too often a program will be approved with- 
out a well-established work breakdown 
structure (WBS) describing the whole pro- 
gram, which inevitably results in large cost 
overruns. The WBS is the basis for the pro- 
curement strategy and often for the manage- 
ment structure. Without it, program 
changes will take place after the contractors 
are in place and have to be paid. Overlaps 
between contracts, as well as missing ele- 
ments and contract changes, are always ex- 
pensive. The following simple rules have to 
be followed: 

1. Each element of the WBS should contain 
a deliverable that can be defined. 

2. The sum of the WBS elements must be 
the total program. (Note that a given pro- 
gram manager may not have the respon- 
sibility for all elements, but they should 
each be defined and allocated.) 

3. Each deliverable should be accompanied 
by a cost and a schedule. The cost should 
include a reserve based on the estimated 
risk associated with that element, and 
the cost should be allocated to that ele- 
ment. 

As simple as these rules sound and as much 
as NASA requires contractors to adhere to 
them, the internal track record is dismal. We 
can go a long way toward containing costs if 
discipline is established early and main- 
tained. 

One last word of caution. A WBS element 
should never be established on the basis of 
function or organization. These elements are 
not end items. Other mechanisms exist for 
identifying these elements, which in general 
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could be defined as program overhead and 
not entirely the responsibility of the pro- 
gram manager. They should be recognized 
for what they are and identified, but they 
should not be included in the WBS. 

Managing the Program 

We have now reached the time in the pro- 
gram when promises have been made, deals 
have been struck, and the program has been 
approved. All that remains is to deliver. A 
custom within NASA stipulates that new 
managers are instilled with the belief that 
the skills required to sell a program and to 
define it are different than those required to 
run it. Certainly some changes can be ex- 
pected, but I believe that such changes are 
better if they occur sometime after a phase 
has been entered and the basic management 
structures have been established. What the 
program needs at this time is ownership of 
the concept, and changes in management 
will usually result in program changes that 
inevitably will lead to increased costs. This 
is particularly true of the systems engineer- 
ing group that has carefully balanced the re- 
quirements against the design and is famil- 
iar with the “why” of a decision as well as 
the “what.” So far the total expenditure has 
been relatively low, but once the contractors 
are onboard and the manpower begins to 
build up, costs can escalate at an alarming 
rate. In a very short time, increases or de- 
creases in performance, extensions or reduc- 
tions in schedule, and decreases in annual 
funding will all increase cost. 

Design to Cost. There is much talk about 
design to cost but very little action, and for 
this there are a number of reasons. Earlier, I 
mentioned that within NASA there is a ten- 
dency to order the three variables by perfor- 
mance first, schedule second, and only then 
worry about cost. So by tradition, cost tends 
to be put on the back burner. One of the rea- 
sons for this is that during the Apollo pro- 
gram, the cost function was transferred to 
the budget and program control groups. In a 


program where the technical problems were 
so difficult and the budgets were ample, this 
was understandable, but this is no longer the 
case. This situation resulted in a shift away 
from making the design engineer account- 
able for cost as well as performance and 
schedule. 

The second problem occurs when the cost is 
not allocated at the WBS element level, 
where it can readily be traded against per- 
formance and schedule and easily traced. I 
believe that cost must be allocated to the 
lowest possible level (a little scary for the 
program manager), but unless this is done, it 
will be impossible to hold the designer ac- 
countable and unlikely that overall costs 
will be held in check. 

The third problem is that in an organization 
that prides itself on technical excellence, it is 
very difficult not to make things a little bet- 
ter; consequently, there are always plenty of 
ideas around. The credo of the systems engi- 
neer should therefore be: “The better is the 
enemy of the good.” 

Design to Life Cycle Cost. Over the past 
decade, the operational costs of NASA pro- 
grams have steadily risen as a percentage of 
total program costs, largely due to the fact 
that programs have a longer life in the oper- 
ational phase. Whereas 20 years ago oper- 
ational costs amounted to no more than 20 
percent of costs, they are now approaching 
half of the NASA budget. It is time to place 
design to life cycle cost on an equal footing 
with design to cost. The dilemma is that a 
design that allows low-cost operations will 
usually demand higher development costs 
and in turn, this means larger front-end pro- 
gram costs. It is essential that the systems 
engineer make these assessments. The sim- 
plest thing for a program manager to do is 
walk away from this dilemma and let the op- 
erations people worry about it later. As this 
is becoming an overall problem for the Agen- 
cy, the ability to make new starts will de- 
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pend on the ability to ensure that a sufficient 
percentage of the budget is available for op- 
erations. 

Unfortunately, it is difficult to get enough 
operations people to participate early in the 
program, but I believe it is essential. Some 
kind of veto power should be established 
when it comes to making design decisions; 
too many program managers do not feel re- 
sponsible for operations costs and perhaps, 
what is worse, are not held accountable for 
it. Let there be no doubt that operational 
costs are unacceptably high. An operational 
concept must therefore be developed early 
enough in the program to have an effect on 
the design process. 

Change Control. Once a program is under- 
way, the program manager’s responsibility 
is controlling change, which is inevitable. 
Earlier I said that you cannot control what 
you have not defined. It is equally true that 
you cannot control changing something that 
is not defined. First know what it is! A com- 
plete WBS with allocated schedule and cost 
is, once again, the key. Change requests 
must not be limited to solving a technical 
problem. They must be accompanied by cost 
and schedule impacts and, just as important, 
life cycle cost impacts. 

In addition, there is always a rippling cost 
impact caused by change. Other WBS ele- 
ments may be affected, including items in 
different contracts or in totally different 
NASA codes, or line items. For these rea- 
sons, change must be assessed at the systems 
engineering level as well as at the WBS lev- 
el. Perhaps the overriding rule is that 
changes should be difficult to approve but 
easy to implement once the decision is made. 

Managing Cost Reserves. A qualified cost 
estimator would not let a program get start- 
ed without making provision for cost over- 
runs or reserve. The many uncertainties in a 
development program make it essential. An 


analysis of past programs allows a fairly ac- 
curate estimate to be made of what is a rea- 
sonable total amount as a percentage of total 
costs, assuming that the programs are simi- 
lar. Determining how and when the allow- 
ance should be allocated is much more diffi- 
cult. One school of thought says that re- 
serves should be held at the highest level in 
the program and applied only to correct un- 
foreseen occurrences. The problem is that 
this tends to bail out poor performers. 

I believe that the reserve should be deter- 
mined based on the perceived risk of the ele- 
ment when the WBS is formulated. The 
manager of the element should then be held 
responsible for the stewardship of the re- 
serve. In order for this to work, some sort of 
reward system must be established for the 
manager who does not spend the reserve. In 
any case, it would be prudent to maintain 
some reserve at the central level for those 
things that cannot be anticipated. Just to 
keep the system honest, a very simple track- 
ing program can be established to follow the 
expenditure of the reserves at the WBS ele- 
ment level after the fact. I would like to see 
an indepth study done on this subject. 

Traps and Pitfalls 

So far we have talked about where cost fits 
into the program management and systems 
engineering processes. TTiere are a few areas 
that may catch the program manager unpre- 
pared and a few ideas that may be used to 
make life a little easier in the future. It may 
not be possible to implement all of them, but 
it is worth a try. 

Buying In. If you are involved in the selling 
of the program, the easiest trap to fall into is 
underpricing the program. Despite stories to 
the contrary, I do not believe that this is a 
matter of deliberate low bidding. Although I 
once heard a distinguished gentleman say 
that we do business the old fashioned way, 
we do underbid and make up on change re- 
quests. The fact is that every program man- 
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ager I have ever met was convinced that he 
or she could do it for less than the past record 
would suggest. Unfortunately, this usually 
involves changing the way we do business. I 
believe that there are less expensive ways, 
but you should tackle this one at your own 
risk and only if you have the support of the 
very top of the organization. The systems en- 
gineer must be the conscience of the program 
manager during this period. 

Design to Budget. Let us assume that we 
have completed a perfect Phase B and that 
everything is in place, including the rate of 
expenditure by year. It is a virtually certain 
that two things will happen. First, with elo- 
quent rationales and spreadsheets by the 
ton, the various element managers will find 
a need to increase their funding allocation. 
One favorite argument will be that the sell- 
ers of the program, who are no longer in 
charge, will be blamed for not understanding 
the problem. In addition, Congress may add 
a requirement or two. 

Second, the budget will be cut in the Agency, 
at the Office of Management and Budget, 
and finally in Congress. At this point, the in- 
tricate patterns of dependency between per- 
formance, cost and schedule begin to un- 
ravel. In the first year, this is not devastat- 
ing because you can always delay bringing 
the prime contractors on board. But by the 
time they arrive, the trap has been set for 
the most insidious form of management, de- 
sign to budget. Unfortunately, a fact of life is 
that very few research and development pro- 
grams have multi-year funding, and annual 
budgets will be less than planned. The net 
effect is that program costs will escalate, and 
enormous pressures will attempt to bring 
down the annual funding. 

The first remedy is to stretch the schedule, 
and the second is to reduce the scope of the 
program. You will no doubt find yourselves 
in this position, and you will receive a great 
deal of advice from the nonparticipants, but 


you should beware of “descoping.” A cursory 
examination of the cost models will show 
that in the manned programs, only 20 cents 
of every dollar go to hardware. (In the un- 
manned programs, the number is closer to 40 
cents.) Once the management structure is in 
place and the contracts have been awarded, 
virtually all of the other costs are fixed or 
very difficult to reduce. Take out all the con- 
tent and the program cost will still be 80 per- 
cent of the estimate! The lesson is that if you 
are forced to remove content, you should be 
sure to take out every cent that is associated 
with that content: prime wraps, nonprime 
wraps, test beds, personnel, and, if neces- 
sary, the kitchen sink. It will be difficult to 
find, but it will be worth the effort. 

If this were a mystery novel, it might well be 
called “The Case of the Missing 80 Percent.” 
Where does it all go, and why is it only 60 
percent for unmanned programs? Much of 
this is valid and accounts for systems engi- 
neering and integration at all levels of the 
program, including test and evaluation, op- 
erations, and many other things. But it also 
accounts for duplication of test facilities, 
overlaps between assignments, management 
style, inefficiencies and a host of hidden 
costs associated with maintaining the insti- 
tutions that are often invisible to the pro- 
gram manager. The systems engineer is re- 
sponsible for ferreting out the good from the 
bad. It is a simple fact that the first one per- 
cent reduction in these wraps (80 percent to 
79 percent) increases the amount of hard- 
ware by five percent (20 percent to 21 per- 
cent)! A 20 percent improvement in the 
wraps (80 percent to 60 percent) results in a 
doubling of the hardware (20 percent to 40 
percent) or cutting the program costs in half 
for the same amount of hardware! “Thar’s 
gold in them thar hills.” 

The UPN System. The NASA budget is pre- 
pared and submitted using a system of 
breakdowns known as the unique project 
number (UPN) system. All parts of the 
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Agency are required to report their annual 
needs on the basis of this system, including 
the program offices. From a program point of 
view, a fatal flaw in this process is the num- 
bering system, which generally describes 
functions rather than end items and is there- 
fore not in consonance with the principles of 
a WBS system. 

It is essential that the program manager be 
able to trace the equivalence of the UPN 
number and its corresponding WBS element. 
This will require a joint effort between sys- 
tems engineering and the program control 
people. Without this traceability matrix, the 
program manager will not know what is be- 
ing asked for or where the money is going. 
Too often the UPN number is perceived as 
directly equivalent to the WBS element, but 
this is very seldom the case unless the WBS 
is not end-item oriented. (The latter happens 
more often than it should.) One way to avoid 
this situation is to make the annual budget 
call for the program using the WBS system 
and then translate it to the UPN system for 
the purpose of aggregating the total NASA 
budget. I have never seen this happen. 

The Cost of Operations. I mentioned earli- 
er that the costs of operations are now about 
50 percent of the NASA budget. This is part- 
ly due to the increase in the operational life 
of a program and to the fact that we have not 
learned to design systems for operability. It 
has not been necessary in the past. It is also 
true that the productivity of the operations 
infrastructure has not been high on the pro- 
gram manager’s list. If we are to reduce total 
program costs, which are vital to the Agency 
and to the program, it is time to strike a new 
level of cooperation between these two nor- 
mally separate parts. The program and the 
systems engineer must assume a large part 
of the responsibility. 

The Institution and the Program 

Although not directly related to the systems 
engineering process, a number of things bear 
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directly on the program and have a major ef- 
fect on the ability to perform the various pro- 
gram functions. These generally concern the 
relationship between the program and the 
institution. NASA was originally estab- 
lished using the resources of the National 
Advisory Committee for Aeronautics, known 
as NACA, an aeronautical research organi- 
zation that was seldom involved in large de- 
velopment programs. The budget was rela- 
tively small, and there were few contractors. 
In fact, all contracts were signed at the 
Washington office, the NACA equivalent of 
Headquarters. 

It quickly became apparent that, in addition 
to the research centers, a development cen- 
ter was needed. Goddard Space Flight Cen- 
ter (GSFC) was established to perform this 
function. This was rapidly followed by the 
Lyndon B. Johnson Space Center (JSC) in 
Houston, the George C. Marshall Space 
Flight Center (MSFC) in Huntsville, and the 
Jet Propulsion Laboratory (JPL) in Pasade- 
na. Almost immediately, GSFC and JPL be- 
came responsible for multiple unmanned 
programs, which were largely contained 
within a single Center, and JSC and MSFC 
became responsible for multi-center manned 
programs. In both cases, program offices 
were established and the Centers provided 
the resources, both personnel and facilities, 
to support the program. 

With the exception of JPL, which is a Feder- 
ally funded research and development center 
and operates outside the civil service system, 
all NASA personnel and basic facilities are 
funded separately from the programs in line 
items known as Research and Program Man- 
agement (RPM) and Construction of Facili- 
ties (CoF). Program-specific facilities are 
funded by the program and these facilities 
are most often operated by support contrac- 
tors, also funded by the program. This sys- 
tem was established so that the programs 
would be managed by government personnel 
who would rotate from program to program 
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and carry their experience with them. This 
worked very well until the late 1960s when 
the budget began to fall rapidly, and there 
was a significant reduction in NASA person- 
nel. 

By the early seventies, both the budget and 
the number of personnel had been cut in 
half, but the number of Centers remained es- 
sentially the same. The cost of maintaining 
the institution could no longer be sustained 
by the RPM and CoF line items. The solution 
was to tax the programs based on the num- 
ber of personnel that were applied to the pro- 
gram. Unfortunately, the program manager 
does not decide how many people should 
work on the program, which, by tradition, is 
the responsibility of the Center director. 
Neither does the program manager partici- 
pate in determining the level of the tax. 
These decisions, again by tradition, are 
made by the comptroller. 

Maintaining the Institution 

Unless the basic system of funding personnel 
is changed, the programs will most certainly 
be responsible for funding some of the insti- 
tutional costs that are not related to the pro- 
gram; the RPM budget will never be allowed 
to grow to compensate for this. The question 
is rather how large the institution needs to 
be to support the program and how that deci- 
sion is made. I mentioned earlier that the 
WBS should represent the totality of the pro- 
gram and should always describe deliver- 
ables; this problem runs counter to that prin- 
ciple. I believe that the solution lies in ac- 
cepting this cost for what it is, negotiating 
the level of tax with the program manager 
for the duration of the program, and taking 
it off the top each year. It may not be control- 
lable in the normal sense, but at least it is a 
known number. 

Personally, I believe that the Agency would 
be better served if the development centers 
were managed using an industrial funding 
system similar to JPL and many other gov- 


ernment facilities, including the Navy labs. 
But until that happens, it will be necessary 
to find some balance between the institu- 
tional and program needs. 

Management Stability 
Every program will change management 
during its life cycle. The common practice in 
NASA has been to make these changes de- 
liberately between phases. It is not uncom- 
mon to see as many as four different manag- 
ers during a program, including a specialist 
in closing off completed programs. The posi- 
tive side to this is that it is possible to match 
the needs of each phase of a program to the 
special capabilities within the Agency. The 
negative side is that each manager has a dif- 
ferent style, each program has different 
management needs, and often these do not 
match when the changeover occurs between 
phases. One way is not always right and an- 
other always wrong, but each is different, 
and changes even in management style can, 
and usually do, increase the cost of the pro- 
gram. The secret then is to stick with a team 
as long as possible, particularly the systems 
engineering team, something that is easy to 
say and difficult to do in these times of de- 
clining internal expertise and increasing re- 
tirements. 

The Tyranny of Experience 
Too often, you will find resistance to change 
in the way things are done. “We have always 
been successful (measured by performance) 
doing it this way, and its very dangerous to 
change winning ways.” “If it ain’t broke, 
don’t fix it.” “You get no credit for an on-time 
failure.” All true and at the same time, de- 
structive to valid new ways of doing busi- 
ness, especially when it comes to introducing 
more efficient or less expensive ways. When 
the space program started, we had no exper- 
ience and what followed was the most inno- 
vative and exciting period in the history of 
high technology programs. But now we have 
all that experience, and it has become a bur- 
den. By all means, you should keep the wise 
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heads around (they may still save you), but 
take advantage of the explosion in new tech- 
nologies and capabilities, which allows for 
things that we could only dream of 30 years 
ago. You should be careful before you intro- 
duce a change, but you should not dismiss it 
out of hand. 

Does It Matter? 

We have been in the civilian space business 
for almost 40 years, and time after time we 
have shown that we can rise to any chal- 
lenge and lead the competition, provided we 
have the resources. Time and time again the 
Federal Government has provided the re- 
sources. We have been the envy of the world. 
We have written the book on the subject, 
both from a technical and a management 
sense. 

Until now, it was enough to know that we 
were the best. There was no established com- 
petition, most of the money was spent inter- 
nally, and cost efficiency was second to per- 
formance. Some have characterized it as a 
Works Projects Administration (WPA) for 
the technologists! The problem is that in this 
era of budget deficits and trade deficits, 
there is not enough discretionary money to 
go around. Even without international com- 
petition, it would be imperative to get more 
out of our research dollars. The trouble is 
that we have learned profligate ways, as nei- 
ther the government nor the contractors give 
rewards for cost efficiency. While we were 


basking in this glory, the rest of the world 
has been catching up; they have read the 
book. The competition, supported by their 
governments, is getting good and fierce. 

But there is a difference; the competition be- 
lieves that the space business is here to stay. 
I said space business, but I meant commerce, 
and in commerce cost efficiency is para- 
mount. Do we still want to stay at the top, or 
are we ready to leave it to the rest of the 
world? Are we prepared to do what is neces- 
sary to stay in the game? After all, it’s only a 
space program. Does it matter? You bet! 

Can Anything Be Done? 

In this paper, I have attempted to show 
where cost fits into the space program’s engi- 
neering and management business. A combi- 
nation of things have placed cost at the bot- 
tom of the priority ladder except in matters 
of the inexorable annual budget. There are 
many ways to improve cost efficiency, some 
of which are available to the program man- 
ager. In the long run, it will take a concerted 
effort by all of us to make a difference. The 
Executive branch and Congress, together 
with industry and academia, must work as 
before, when we perceived that we were sec- 
ond. In the meantime, I hope that I have 
been able to give the budding systems engi- 
neer and program manager a few tips to do 
something about the problem of cost consid- 
erations. We can only do something about it 
if we want to! 
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The strategic plan for NASA’s new explora- 
tion initiative begins: 

On July 20, 1989, the President of the 
United States committed the nation to a 
major initiative to explore space. The 
goal of this initiative is human explora- 
tion of the Moon and Mars as soon as 
possible within the constraints of nation- 
al resources. 

From several years of studying alterna- 
tive strategies and debating the relative 
merits of national investments in space 
exploration has emerged a consensus; 
i.e., that expanding human presence and 
activity beyond Earth orbit is an appro- 
priate and inevitable long term focus for 
the nation’s space program. 1 

The plan states three strategic themes: incre- 
mental, logical evolutionary development; 
economic viability; and excellence in man- 
agement. All of these intricately involve the 
cost estimation process, and, as will be 
shown, will be completely dependent upon 
the engineering cost estimator for success. 

The purpose here is to articulate the issues 
associated with beginning this major new 
government initiative, to show how NASA 
intends to resolve them, and finally to dem- 
onstrate the vital importance of a leadership 
role by the cost estimation community. 

The Demand for a New Management 
Paradigm 

The exploration program objective, as stated 
in the NASA Strategic Plan, emphasizes ear- 
ly accomplishments, but also recognizes that 
the environment today is substantially dif- 
ferent, and that whatever is done must be 
done within the limits of realistic budgets. 


This presents a double challenge to NASA, 
where the length of a human mission space- 
craft development program has approached a 
decade. For a new era of space exploration to 
begin at all, it is believed, early milestones 
must be set which are challenging and at- 
tractive to those who must provide program 
resources (the National Space Council, OMB, 
Congress), oversight bodies which have sent 
strong signals that multibillion dollar explo- 
ration programs requiring decades to reach 
fruition are not in NASA’s future. NASA 
must therefore provide early, visible, worth- 
while milestones in exploring space. 

At the same time, costs must be reduced. 
Generally, compressing a given task into a 
much shorter period of time will greatly in- 
crease the annual funding required. NASA 
must find ways to compress and at the same 
time lower annual funding requirements. Al- 
ready, the experienced engineer/estimator 
will begin to have concerns. Accomplishing 
these challenges one at a time is difficult 
enough, but to accomplish both at once is be- 
yond the paradigm of conventional aerospace 
program management, on which almost all of 
our estimation methods are based. 

Another example of the problems with the 
conventional aerospace management para- 
digm is illustrated by the case of the Space 
Station program. That program struggled to 
lower annual budgetary requirements by re- 
ducing the mission content of the program; 
but the more the program is modified, the 
more changes are incurred, the higher the to- 
tal program cost, and the more pressure on 
annual budgets. Reducing content can 
achieve the most significant cost savings be- 
fore the program is started. Once underway, 
content reductions often exacerbate an al- 
ready bad cost situation. 
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The situation is made even worse by the ab- 
sence of good tools. Every cost model em- 
ployed by the aerospace industry today (with 
perhaps one or two exceptions) merely pre- 
dicts the future based on the behavior of the 
past. If, then, extrapolation of past behavior 
will not produce the desired result, how can 
cost models based on that behavior serve us 
at all? NASA has learned that they cannot. 
In fact, they can become the tools of those 
who oppose new programs, to prove to the 
Congress and the Administration that un- 
dertaking of the grand new adventure is fol- 
ly. Of course, the point would not have been 
proven at all, but the perception of proof is at 
least as powerful as proof itself. One can look 
for situations of this nature to arise within 
the next few years. The cost estimator, then, 
can become the enemy of progress. Or, as will 
be demonstrated, he or she can lead the way 
to change. 

Struggle as we will within this old paradigm, 
we will not be able to resolve the dual chal- 
lenges of lowering annual costs substantially 
while significantly reducing program length. 
Impossible. So, what can be done? Should 
NASA simply go the President and admit de- 
feat? To do that would probably doom what 
remains of the human adventure in space, 
and not only jeopardize future programs, but 
raise the question of the continuation of cur- 
rent programs as well. 

When a problem cannot be resolved within 
one paradigm, it is obviously necessary to 
change to a new one. But before that new 
paradigm is defined, a direction must be es- 
tablished, and a model created for the new. 
To change course without a new destination 
would be equally disastrous. The research 
done by NASA to identify that new model fol- 
lows. 

A Summary of the Cost Challenges 
Facing Exploration 

Much of the planning of any new venture in- 
volves matching demands for resources with 


the predicted supply. Within the old para- 
digm, the supply of resources has often been 
predicted only by estimating the demand. In 
former times, this process has worked be- 
cause the aerospace and defense industries 
have generally received ample support from 
the nation to create this norm. 

However, the norm is today being threat- 
ened. As each new human space venture 
since the initial Apollo lunar landing has 
been launched, the availability of resources 
has become increasingly scarce. The Space 
Shuttle, a program designed to lower the cost 
of placing humans and cargo into space, de- 
feated its own raison d’etre when it was 
forced, for reasons of annual budget limits, to 
eliminate its completely reusable booster, 
and to limit the availability of on-board 
autonomy which would have reduced the ex- 
pense of ground control and checkout. 

Similarly, Space Station Freedom was beset 
from the outset with mission compromises 
caused by annual budgetary limits, and be- 
came a much less capable facility than was 
originally conceived by NASA. Each year the 
program suffered further and further content 
reductions in an attempt to meet annual cost 
constraints. 

Today’s situation has found the nation even 
less able to pay for large, new manned space 
programs than ever before in our spacefaring 
history. But it has taken some time for this 
realization to influence the program plan- 
ning paradigm. For example, as recently as 
1991, the Advisory Committee on the Future 
of the U S. Space Program was making rec- 
ommendations to the NASA Administrator 
which, while recognizing that there were 
budgetary constraints, were predicated on 
the availability of greatly increased NASA 
budgets (see Figure 1). 

As NASA began its studies of human explo- 
ration missions under the old management 
paradigm, the cost models employed pro- 
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Figure 1. Budget Assumption of the Advisory Committee 
on the Future of the U.S. Space Program 


duced estimates such as those shown by the 
middle curve of Figure 2. Overlaid with the 
budget projections of the Advisory Commit- 
tee on the Future of the U.S Space Program 
(top curve, Figure 2), there seemed to be no 
reason to doubt that exploration had a bright 
future. 

However, when better budgetary estimates 
were made with econometric models and 
with full understanding of the true likeli- 
hood of NASA budget growth (lower curve of 
Figure 2), the dilemma became apparent to 
some for the first time. 

Four Things Which Must Be Done 

To resolve the dilemma of budget growth, 
NASA must do four things. First, full atten- 
tion must be paid to the mission statement of 
the opening paragraph: “to the Moon and to 
Mars as soon as possible, within the con- 
straints of national resources.” With a focus 


on the purpose of human exploration, the 
need for much of the content of previous 
planning exercises can be questioned, and 
missions constructed which contain only 
mission -related items. 

Second, existing NASA and other govern- 
mental resources must be found and lever- 
aged. For example, much of the money cur- 
rently being spent by NASA on science and 
technology is fully applicable to the purposes 
of human exploration; however, some mis- 
sion focus must also occur in these areas. The 
use of other federal resources can include the 
use of national laboratories and DoD assets, 
and these are being investigated. 

Third, NASA must implement a new man- 
agement paradigm which does things faster, 
smaller, and less expensively, using the 
enormous cost leverage which results from 
cultural change. 
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And finally, some new resources must un- 
doubtedly be found for NASA. These will be 
much more likely to be forthcoming if the 
Agency once again gains the full confidence 
of the Congress and other oversight groups 
by demonstrating competence and efficien- 
cies associated with the change to the new 
management paradigm. 

Benchmarks for the New Paradigm 

NASA is conducting research to identify 
benchmarks, guidelines, and processes for 
the low cost, short schedule paradigm. Ex- 
tensive interviews have been conducted and 
analyses performed to identify high technol- 
ogy programs which have been done under 
different management norms, and which 
have resulted in high performance, low cost, 
quickly developed products. 

Results of the interviews, summarized in 
Figure 3, indicate a wide consensus on the 


part of those successful managers inter- 
viewed, that NASA should confine itself 
more to the development of good, perfor- 
mance-based requirements, and establish a 
more arms-length relationship with the pri- 
vate sector to allow the power of the competi- 
tive marketplace to produce excellent pro- 
ducts. 

Historically, the highly interactive relation- 
ships between NASA and its contractors 
have produced excellent products, but pro- 
gram change rates have been in the thou- 
sands per year, and high costs and long de- 
velopment schedules are typical. Contractor 
awards should be based on the performance 
of products as demonstrated in mission per- 
formance. Taken altogether, the findings of 
this research, as summarized, provide very 
useful benchmarks for designing future pro- 
gram management processes. But do these 
findings describe a feasible set of conditions? 
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Another part of the NASA research has dealt 
with the identification of programs which 
have been accomplished more under the de- 
scribed new set of conditions than under the 
existing aerospace paradigm. Programs like 
the XR-71, the F-117, and the YF-16 (Lock- 
heed, Lockheed, and General Dynamics, re- 
spectively) have demonstrated that high 
technology programs can be done very quick- 
ly (all of these programs produced flying air- 
craft in approximately two years) and at 
costs significantly below those which would 
have resulted from the old paradigm. 

However, much more work is needed by the 
industry to plan and execute an orderly tran- 
sition from one culture, one paradigm, to the 
new paradigm for NASA space exploration. 
The cost estimator can play a key role in the 
process. 


Cost and Culture: The New Calculus 
of Cost Analysis 

Cost estimation methods employed by the 
aerospace industry for program planning are 
usually parametric in nature, although some 
detailed estimating is used for special pur- 
poses which do not readily lend themselves to 
performance or size-based parametrics. 

In most parametric estimation, for reasons 
that parametric estimators seek the best pos- 
sible analogies from their historical data- 
bases, the implicit assumption is that a new 
program will be a product of basically the 
same cultural and management conditions 
(the same paradigm) as programs of the re- 
cent past. However, when this assumption is 
made for exploration programs, the resulting 
estimates exceed realistic budgetary expec- 
tations. 


The ingredients of successful low-cost, high technology programs are well known 
and universally recommended by successful program managers interviewed 

- Use government only to define requirements 

- Keep requirements fixed: once requirements are stated, only relax them; never 
add new ones 

- Place product responsibility in a competitive private sector 

- Specify end results (performance) of products, not how to achieve the results 

- Minimize government involvement (small program offices) 

- Insure that all technologies are proven prior to the end of competition 

- Utilize the private sector reporting system: reduce or eliminate specific 
government reports 

- Don't start a program until cost estimates and budget availability match 

- Minimize or eliminate government imposed changes 

- Reduce development time: any program development can be accomplished in 3 
to 4 years once uncertainties are resolved 

- Force people off of development programs when development is complete 

- Incentivize the contractor to keep costs low (as opposed to CPAF, CPFF of NASA) 

- Use geographic proximity of contractor organizations when possible 

- Use the major prime contractor as the integrating contractor 

Figure 3. Benchmarking Lessons Learned from Interviewing Successful Program Managers 
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Therefore, one must conclude that, if major 
exploration programs are to be performed, 
significant cultural change to a new para- 
digm is an absolute necessity. However, ex- 
cept for the G.E. PRICE series of models, the 
aerospace industry cost estimator is not 
equipped to deal quantitatively with cultural 
change as an explicit variable using the ex- 
isting tools and databases. It has, therefore, 
been necessary to construct a new type of cost 
model, which, instead of predicting costs 
from “technical” and performance param- 
eters, will predict the cultural levels required 
to produce a given cost outcome. 

Working with the then-RCA PRICE Systems 
organization, NASA performed a study to de- 
termine the effects of various culturally im- 
posed standards on costs. The study results2 
demonstrated conclusively that, while there 
is correlation between cost and such things 
as government-imposed parts traceability re- 


quirements, major differences still exist in 
program costs which can only be explained 
by the organizational “manner of doing busi- 
ness,” or culture of the developing agent. 

These results have recently been repeated by 
Kelley Cyr of NASA’s Johnson Space Center 
and employed in the development of a new 
series of cost models. Figure 4, based on a 
statistical analysis of several hundred points 
of data, portrays the quantified relationship 
between cost and development culture. 

In the current environment, this type of cost 
equation provides the needed utility to relate 
costs to program management and manufac- 
turing culture. Particularly in government 
aerospace product acquisitions, the highest 
levels of product performance have been for 
over a generation the object of most develop- 
ment efforts in the industry. This has created 
a culture where program cost, while highly 



Figure 4. Effect of Development Organization Type on Program Development 

and Production Cost 
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important, has not generally been treated as 
a critical design parameter. 

In this climate, it is not surprising that many 
design engineers and program planners are 
generally not well equipped to deal with cost 
as an explicit parameter. Who, then, is avail- 
able to provide the leadership which will be 
so vital to the conversion of our industry to 
lower cost, shorter schedule norms? 

Who is closest to the necessary data? Who 
has the best understanding of the dynamics 
of engineering processes as they are influ- 
enced by costs and schedules? Who (often) 
has training in both engineering and busi- 
ness practices? Who is most often the one 
who bears a major responsibility for any ma- 
jor cost reduction activity? It it proposed here 
that there is no one better equipped than the 
company cost estimator. The case can be 
made that the cost estimator is best able to 
provide answers to all of these questions. 

In the case of the exploration initiative, the 
activities of the cost estimation team will 
have the most significant influence of any on 
the future success of the venture. That team 
must not only develop compelling cost esti- 
mates, but they must also lead the way in 
providing the rationale, the supporting argu- 
ments, to provide cogent reasons why NASA 
can truly accomplish what it proposes (such 
as returning humans to the Moon by 1999) 
within the available budgets. It is also the 
cost estimation team who must provide the 
information for the design teams to utilize in 
developing requirements for low-cost, early 


missions. They may be the only team who 
can complete the bridge to the new para- 
digm. If they fail at this, the entire venture 
will probably fail to be accepted by the Con- 
gress and the Administration. 

The aerospace industry cost estimating com- 
munity holds the future of the United States 
Space program in its hands. While this com- 
munity is not unto itself sufficient to develop 
the new initiative, it is vitally necessary. 

Conversely, the cost estimating community 
is totally sufficient to prematurely end the 
life of American space exploration, at least 
for this generation. It is far easier to develop 
strong arguments for why the nation cannot 
afford to send humans to the Moon and to 
Mars than it is to prove that it cannot afford 
not to do it. It is far more comfortable to fall 
back on that which has served us well in the 
past and hold to the old culture, to stay with 
the old paradigm. It is far easier to use our 
existing, culturally-bound costing methods 
than it is to seek methods which can point 
the way to changes that may brighten the fu- 
ture of our entire profession, if not our indus- 
try. 

The job of cost estimators has never been 
easy. The results of their work have often de- 
termined whether or not their company wins 
or loses a major competition. But today, it is 
the cost estimators who wield the enormous 
power of life or death over the future of the 
United States space exploration program. It 
is earnestly hoped that this awesome respon- 
sibility will not be taken lightly. 
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by William C. Keathley 

Personal experiences in the management of 
projects and shared experiences with col- 
leagues have convinced me that a Cost Plus 
Award Fee contract is the best procurement 
vehicle for the high-tech, one-of-a-kind, de- 
velopment projects that constitute most of 
NASA’s projects. But, like most things, suc- 
cess isn’t automatic. It takes work to make it 
happen, and the successful implementation 
of award fee contracts is no exception. In fact, 
the use of this type of contract requires more 
government and contractor effort than other 
forms of contracts. But, in my opinion, it’s 
worth every hour spent. 

Over the years, I’ve collected a list of “lessons 
learned” related to the use of award fee con- 
tracts. I’ll try to articulate those lessons ade- 
quately in the following text. Keep in mind 
that I’m not speaking from the standpoint of 
a procurement officer. My observations come 
from the day-to-day use of these contracts in 
various positions I’ve held — project manager, 
director of flight projects (project manager’s 
supervisor), and fee determination official. 

An award fee contract is described as an ar- 
rangement whereby the government periodi- 
cally awards a fee consistent with the cost, 
schedule and technical performance that is 
achieved by a contractor during a preset peri- 
od with preset award fee pools. 

Rationale 

Let me explain why I like award fee contract- 
ing. First, it’s the only contracting method 
where both government and contractor goals 
are closely linked. The government wants 
cost, schedule and technical performance; the 
contractor wants profits. The better the total 
performance, the better the fees (profits) will 
be. Compare that with a fixed price contract 
where the total price (cost plus fee) is fixed. If 


the cost of a fixed price effort is underesti- 
mated, the contractor may sometimes make 
adjustments that impose risks to the techni- 
cal performance. This protects the contrac- 
tor’s profits but imposes risk on the govern- 
ment’s goal for technical performance. Other 
ways exist for contractors to protect their 
fees in a fixed price arrangement (all of them 
bad for the government), but that subject de- 
serves a separate paper. 

Second, an award fee contract has a built-in 
mechanism to conveniently alter and empha- 
size program events in order to satisfy cur- 
rent external and internal situations — and 
the government is involved in these adjust- 
ments. Prior to each award fee period, the 
government and contractor project managers 
review the plan for the upcoming period, 
agree on the planned events, and place the 
appropriate emphasis on each event. Should 
problems arise (and they always do), the plan 
and the fee emphasis can be adjusted accord- 
ingly. This is considered by most project 
managers to be the most important feature of 
award fee contracts. 

And while Fm on adjustments, I’d like to 
mention the use of “rollovers,” in which lost 
fee from prior periods is used to “sweeten the 
pot” on future events that have become so 
critical that additional emphasis is warrant- 
ed. Rollover is a powerful award fee tool to 
motivate contractors if used properly. 

Third, the award fee process demands good 
communication between the government and 
contractor participants. And every project 
manager knows or should know that good 
communication is a necessary ingredient of 
every successful project. The meetings re- 
quired by award fee contracting reinforce the 
need for clear communication. 
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Fourth, I have learned that contractor per- 
formance on award fee contracts is superior 
to performance by the same contractors on 
other types of contracts. The quality of the 
product is certainly superior. The fee earned 
by those contractors is better than they could 
have received on other cost type contracts, 
and it should be. Remember: better perfor- 
mance, which the government wants, results 
in higher fees, which the contractor wants. I 
don’t have any data on fixed price contracts 
because there is no government knowledge of 
final costs of those types of contracts. But I’ll 
bet award fees are close to the profits custom- 
arily realized by contractors, even on fixed 
price development contracts. 

The downside to award fee contracting is the 
additional contractor and government per- 
sonnel required to implement award fee con- 
tracts. It is certainly true that more people 
are needed to formally assess contractor per- 
formance, conduct performance evaluation 
board meetings, and report findings to the 
fee determination official. But I maintain 
that most of that work should be done under 
any circumstances, and the improved com- 
munication is worth the effort. So I’m not 
sympathetic to those complaints. 

Implementation 

All the good features discussed above can go 
down the drain with faulty implementation. 
I’ve found the following nine ground rules to 
be effective in properly implementing the 
award fee contracts in which I’ve been in- 
volved. I will readily admit that there should 
be many ways to skin this cat, but frankly, 
I’ve found no effective alternatives to the fol- 
lowing rules. I’ve also seen instances where 
both the government and the contractor 
failed to reach their objectives as a direct re- 
sult of deviations from one or more of the fol- 
lowing rules. 

First, the government project manager must 
chair the Performance Evaluation Board 


(PEB). After all, the project manager is the 
key official selected by NASA to be responsi- 
ble for the project cost, schedule and techni- 
cal performance. The project manager is in 
the best position to evaluate the importance 
of the performance during the project evolu- 
tion and obviously has the most to gain or 
lose from that performance or lack thereof. If 
that’s not true, the Agency should find an- 
other project manager. On the other hand, 
it’s crucial that the contractor understand 
that the government project manager is the 
most influential government individual for 
all project activities, and looking elsewhere 
for project-level influence is unproductive. 

Second, the PEB should consist of institu- 
tional members who are participating in the 
project: procurement, business (program con- 
trol in some Centers), engineering, and prod- 
uct assurance (quality control and safety at 
some Centers). Depending on the end item or 
service, science and operations should also be 
added. It’s advisable to keep the PEB mem- 
bership as small as possible, and it’s impor- 
tant to select individuals with experience ap- 
plicable to the end item or service delivered. 
In other words, make sure they are capable of 
understanding what the contract monitors 
are telling them. 

Third, the Fee Determination Official (FDO) 
should be no higher than one level above the 
project manager and, in fact, should be the 
project manager's line supervisor. The FDO 
must have more than a passing knowledge of 
the project’s status. This requires frequent 
interactions with the project manager, which 
the supervisor’s position provides. Devi- 
ations from this rule can result in some aw- 
fully dumb fee determinations. I might add 
that if the project manager reports to the 
Center director, the deputy Center director 
should be the FDO. Center directors should 
not be FDOs and should be reserved to re- 
solve institutional or project issues should 
they arise. 
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Fourth, use adjectives that can be understood 
and that properly describe performance lev- 
els. I prefer the academic model where “Sat- 
isfactory” is used for barely passing perfor- 
mance (a 60 or 70 percent performance rat- 
ing, depending on your preferences.) Levels 
below “Satisfactory” can be identified as 
“Poor” and “Failing.” Levels above “Satisfac- 
tory” can be called “Good” and “Excellent.” 
It’s confusing to everyone when fee curves 
are set so that the fee letter indicates a con- 
tractor got a “Superior” rating but received 
only 65 percent of the available fee for that 
period. Don’t laugh; that’s actually hap- 
pened. 

Fifth, skew the fee curve (fee earned vs. per- 
formance rating) so that most of the avail- 
able fee falls above “Satisfactory,” or what- 
ever you’ve decided to call passing perfor- 
mance. This clearly shows our desire for high 
performance and motivates the contractor to 
exceed a mere passing grade. 

Sixth, make the award fee periods sufficient- 
ly long to allow time to correct deficiencies 
after a midterm review by the project manag- 
ers. I prefer six-month periods. This allows 
the project managers to assess the perfor- 
mance status three months into the period in 
order to identify performance problems, and 
then still provides three months to correct 
the situation before final evaluation and 
scoring of that period’s performance. Periods 
of less than four months preclude this impor- 
tant process. 

Seventh, offer contractors an opportunity to 
present self-assessments of their perfor- 
mance to the PEB and the FDO. Some con- 
tractors will choose not to do this, but the in- 
vitation ought to be given. If the offer is ac- 
cepted, I believe the PEB should hear the 
contractor’s self-assessment before making 
the final rating. As an FDO, I definitely pre- 
ferred hearing the contractor’s self-assess- 
ment before hearing the PEB’s story. I’ve 
found that the major advantage of contractor 


self-assessments is that they indicate faulty 
communication between the government and 
the contractor — which will kill a successful 
project more quickly than anything I know. 

Eighth, rollovers should be allowed in the 
award fee plan but never promised. They 
should be left to the discretion of the FDO 
and result from recommendations by the 
PEB. They should be used infrequently and 
always targeted to specific events that have 
become crucial to the success of the project. 
Specific “go/no-go” performance criteria 
must be established for these events and an- 
nounced in the fee letter for the period pre- 
ceding the period in which the selected event 
falls. 

Finally, and most importantly, the contrac- 
tor project manager and the government pro- 
ject manager must jointly agree on miles- 
tones and criteria, and the emphasis to be 
placed on each, before the beginning of each 
award fee period. And then everyone must 
stick to the agreements. This won’t eliminate 
disagreements with the amount of fee award- 
ed, but it does eliminate surprises, which are 
simply unacceptable. Nothing can kill an 
award fee process quicker and demoralize 
contractors more than to be “dinged” for 
something they didn’t know. 

Fee Determinations 

Now let’s look at the lessons learned in the 
awards themselves. The first and most im- 
portant ground rule is: don’t play games. If 
the contractor earned all of the fee, by all 
means award it. Don’t fall into the trap of 
telling yourself, “If I give 100 percent, the 
contractor will start expecting it every time.” 
Or: “The contractor earned 100 percent, but 
Til give 80 percent to give some room to im- 
prove.” Or just as bad: “If I give the contrac- 
tor the 20 percent really earned, I’ll get the 
project manager fired.” Awards that are too 
high or too low are equally bad. Awards that 
are too high tell the contractor to under per- 
form and get away with it. Awards that are 
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too low tell the contractor that no matter how 
hard the work and how much the accomplish- 
ment, efforts will be in vain. Both situations 
are bad and will demoralize the contractor. 
Stick to the prior agreements and award the 
fee consistent with the actual performance. 

If the performance is deficient and your 
awards are consistently fair, you’ll soon see 
the performance improve. If the performance 
is good, and the contractor is convinced that 
fees will be lost by backsliding, the perfor- 
mance will remain high. In case you didn’t 
notice, the operating word is fair. By the 
way, it’s a good idea to keep histograms for 
the percentage fee earned as the program de- 
velops. If the awards have been consistent 
(fair), you’ll see the hills (good times) and 
valleys (problems) that occur in any develop- 
ment activity. 

Award Fee Letter 

Now for the important fee letter where you 
tell the contractor about the determination. 
Believe me, you can ruin a good award fee 
process and all the work you’ve done by issu- 
ing an award fee letter that no one under- 
stands. It would be impossible to overstate 
the importance of these letters. 

I’ve found the letters should have four basic 
parts. The first paragraph is really a boiler- 
plate paragraph that references the contract 
title and number, identifies the period for 
which the award is given, states the percent- 
age of the award earned and the specific dol- 
lar amount, and gives the performance adjec- 
tive rating. The second paragraph should 
identify the instances of commendable per- 
formance. Be specific, even if you have to use 
bulleted items. Be clear. The contractor must 
understand which ratings were high so as to 
pass the accolades along to the working 
troops. The third paragraph should identify 
deficiencies. Again, it’s extremely important 
to be specific and clear. I call the final fourth 
paragraph the “message” paragraph. The 
content of this paragraph can range from 


“keep up the good work” to “be advised that 
continued inferior performance in (a certain 
area) will have serious effects on future over- 
all fee determinations.” 

A good contractor general manager will do 
several things with the fee letter; that is, if it 
is understood. First, a meeting with the pro- 
ject manager will be held to review the letter. 
The project manager will be commended for 
the things done properly (second paragraph), 
actions will be identified to correct recur- 
rence of the deficiencies (third paragraph), 
and the message (fourth paragraph) will be 
discussed and actions (project or institution- 
al) will be identified to respond to the thrust 
of the message. 

Next, the good general manager will send a 
letter to the FDO stating that the award has 
been reviewed with the project manager, the 
recognition of the commendable items is ap- 
preciated, the deficiencies and message are 
understood, and appropriate actions have 
been assigned. In addition, the general man- 
ager will now be in a good position to report 
the profit status on this contract and articu- 
late the details of the award. All of these 
good things transpire when the contractor 
understands the fee letter. Otherwise, there 
is no follow-up or feedback, the situation can- 
not be explained to corporate reviewers, and 
everybody loses. 

The understanding of the awarded fee is so 
important that I added one more step to the 
process. As an FDO, if a general manager 
called and verbally complained about certain 
elements of the award, I would discuss the 
call with the government project manager 
and provide verbal feedback to the general 
manager. If the complaint came in writing, I 
would reconvene the PEB with instructions 
to draft a written response to only the specific 
concerns stated in the general manager’s let- 
ter, not every element of the award. I would 
then discuss the recommended government 
response with the PEB. If I agreed with the 
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PEB position, I would send the written re- 
sponse to the general manager. I have 
changed a prior award in the contractor’s fa- 
vor after learning that the PEB used errone- 
ous information. In that case, the general 
manager was correct and the contractor 
earned the fee increase. After all, that was 
the fair thing to do. The contractor response 
to that small dollar change was tremendous, 
and performance improved markedly. 


So in summation, I believe that award fee 
contracting is particularly suited to the one- 
of-a-kind development projects which consti- 
tute most of NASA’s efforts. I do not believe 
fixed price contracts or fixed price plus incen- 
tive contracts belong in this environment. 
Perhaps someone else may wish to argue the 
advantages of the latter types, but my exper- 
ience suggests that award fee contracting is 
the better way to go. 
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A Costand Performance System (CAPS) 
in a Federal Agency 

by W.F. Huseonia and P.G. Penton 


The John C. Stennis Space Center (SSC) in 
southern Mississippi serves as NASA’s pri- 
mary facility for propulsion testing. This in- 
cludes testing stages and propulsion systems, 
for the Saturn V moon rocket during the 
Apollo lunar landing program, and current- 
ly, for the Space Shuttle. In addition, SSC is 
building new test facilities to advance turbo- 
machinery technology for propulsion systems 
of future generations of vehicles. 

SSC also has capabilities in remote sensing, 
Earth sciences and applications develop- 
ment. SSC is NASA’s lead Center for the 
commercial development of space remote 
sensing technology and has substantial work 
under way in the transfer of technology to 
the public sector. SSC also provides technical 
and institutional support services to 18 other 
Federal and state agencies in residence. 


Although our principal mission for NASA is 
to support the development and acceptance 
testing of large propulsion systems for the 
Space Shuttle, National Launch System 
(NLS) and the Advanced Solid Rocket Motor 
(ASRM), the remaining missions of the Cen- 
ter foster cooperative research and develop- 
ment activities that serve to broaden our un- 
derstanding and management of our natural 
resources. These missions are described in 
Figure 1. 

SSC is organized in a classical government 
functional project structure. The functions of 
Legal, Public Affairs, Personnel, Resources, 
Procurement and Safety are shown in Figure 
2. There are two project offices (ASRM and 
NLS) that carry out their responsibilities in 
a matrix organization with the line director- 
ates. The three line directorates (Propulsion 



• Provide, manage and operate 
facilities, laboratories and 
related capabilities essential 
to the development of large 
propulsion systems including 
SSME, NLS and ASRM 


Source: (1988 SSC Strategic Plan) 


• Conduct research and devel- 
opment in propulsion test 
technologies, including cryo- 
genics, high-pressure gas, 
methodology, engine diag- 
nostics and safe operations 

• Conduct research and technol- 
ogy development in Earth and 
environmental system sciences 
and observations, commercial- 
ization of remote sensing and 
applications development 


Manage the Center and 
provide technical and 
institutional support to 
SSC programs and to 
resident agencies 




Figure 1 . SSC Roles and Mission 
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Figure 2. SSC Organization 


Test, Science and Technology Laboratory 
and Center Operations) are functionally 
compatible with the SSC mission. 

This organization structure, a hybrid of func- 
tional and matrix, mixed with the fact that 
the missions of the center are primarily ac- 
complished by support contractors, makes 
the tracking of cost, schedule and perfor- 
mance a complex and difficult (yet necessary) 
management function. 

Purpose 

In the late 1970s, the NASA Administrator 
established a study to examine NASA project 
management and to make recommendations 
on how to improve the Agency’s performance. 
A major conclusion of the study was “poor 
tracking of contractor accomplishments 
against approved plans in a timely fashion 
leads to late identification of problems” 
(Hearth, 1991). 

This paper describes a systematic approach 
to aligning funding sources with cost plan- 


ning and performance scheduling. This sys- 
tem establishes a monthly reporting status 
structure, correlates resources with schedule 
and performance, and assesses “what-ifs” 
and their alternatives for management re- 
view (Sneed, 1991). The Cost and Perfor- 
mance System (CAPS) provides the founda- 
tion for successful project management, func- 
tional organization management and over- 
sight of the total organization’s performance. 

This system will be described in generic 
terms with results and implications from its 
specific use directly relating to the Science 
and Technology Laboratory (STL) at SSC. 

Planning Phase 

The model depicted in Figure 3 begins with a 
fund source and its identification within the 
federal financial system as a Unique Project 
Number (UPN). The UPNs are assigned ac- 
cording to the NASA Headquarters organiza- 
tion and program structure and tracked at 
various levels in the organization. For exam- 
ple, NASA may assign a three-digit UPN 
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XXX for an overall project fund source. The 
project may then be divided into subsets, 
each of which will be assigned a five digit 
UPN XXX- YY. These subsets may be further 
divided and funded with a seven-digit UPN 
XXX-YY-ZZ. Figure 3 represents the generic 
flow in the planning, implementation and 
analysis of cost and performance within STL 
at SSC. 

These funding sources (UPNs) represent the 
dollar amounts assigned to the project from 
the various headquarters organizations. It 
further represents the overall results of the 
budgetary cycle within the organization or 
agency for the fiscal year. Thus, given the 
dollar amounts assigned to the UPNs, the 
project planning cycle can begin. Cost plans 
for individual projects are developed for spe- 
cific elements of cost and spread across the 


fiscal year. A typical cost plan is shown in 
Figure 4. 

The elements of cost are the classical labor, 
materials, equipment and other direct costs 
(ODC). Also included are the subsets of labor 
(straight time, overtime, etc.), subcontractor 
cost and the subsets of ODC (travel, training, 
bases, etc.). The plan “cost elements” and the 
report of actual costs by the contractor 
should be completely compatible. The cost 
plan should reflect the manpower required 
each month, the anticipated cost of equip- 
ment or materials and projected travel and 
training and other costs as listed. The cost 
plan is, at best, just a plan. It should have 
adequate schedule slack and budget contin- 
gency to solve inevitable problems along the 
way (Longanecker, 1990). It and the ele- 
ments of cost can be and should be adapted to 


64 






















A COST AND PERFORMANCE SYSTEM IN A FEDERAL AGENCY 


the types of cost one may incur in any specific 
project. There may also be unique costs. 
Within STL, for example, aircraft missions 
flown to collect remote sensing data for scien- 
tific research and development projects are 
one important cost that is planned, tracked 
and reported by either the contractor or gov- 
ernment personnel as “aircraft missions.” 

At the top left of the STL Project Plan is the 
UPN number (551-20-00) for this project. 
The UPN may support more than one project; 
therefore, this project is assigned a benefitor 
code (BF) of VVB. Each project has assigned 
a benefitor code which is directly related to a 
UPN. The next item on the top left side is the 
project name (SIDS/IDS). SIDS/IDS stands 
for the Shuttle Ice Detection System, a sub- 
set of the Infrared Detection Systems being 
developed at SSC. The final description is the 
crew/depart. The Technology Development 
Division within STL is responsible for the 
project and has a department number of 
HA20 designated by 20 on the form. 

On the top right side is the financial status at 
the beginning of the fiscal year when the 


plan is initially developed; the uncosted 
carry-over, new obligation authority, total 
available to cost and planned carry-over. 

The actual work tasks represented in the ele- 
ments of cost are scheduled in the perfor- 
mance milestone plan. Figure 5 is a typical 
milestone chart for a particular project. The 
project tasks are listed, and expected comple- 
tion dates are spread across the fiscal year by 
month. 

The cost plans and milestone schedules are 
necessarily prepared at the beginning of the 
fiscal year. They are the basis for planning 
resources across the organization. This in- 
cludes, but is not limited to, personnel staff- 
ing, skill mix, procurement plans, training 
budgets, travel budgets and office space re- 
quirements. Actual monthly costs are com- 
pared with the plans and any significant 
variance ( ± 10%) is explained. Knowledge of 
this variance and the explanation for it allow 
the project manager to make the necessary 
adjustments to keep the project on schedule 
and within budget. Therefore, the first step 
in CAPS is planning. 
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Figure 4. Project Cost Plan 
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A Exp«ct»d Data 
A Complatad 
A-A Sllppad to . . . 


SIDS/IDS 


Protect: 

UPN/BF Coda: S51-20-00/WAQ 


Explanation 


Complete Latxxaiory Tests 
Complete Test Report 
Compleie final Configuration Definition 
Upgrade System to final Configuration 
Complete final Report 


Oct 


Nov 


Dec 


I 


Jan 


Fab 


Mar Apr 


May 


Jun 


Jul 


Aug 


Sap 


What Is primary objective of this project (or deliverable)? loe Detection System Proto type Definition 


Figures. Milestone Schedule 


The Science and Technology Laboratory has 
in any one fiscal year 60-65 fund sources 
(UPNs) at the seven-digit level. These are 
broken out into 164 benefitor codes (separate 
projects) which are planned and scheduled in 
detail by 40 project scientists and engineers. 

The planning process starts approximately 
three weeks prior to the end of the fiscal year 
(September) and is completed one week prior 
to the first fiscal month (October). The cost 
plans and milestone schedules are formulat- 
ed and put in place to track cost and perfor- 
mance. At STL, they remain unaltered (not 
changed) until midyear. At midyear the 
plans are updated to project the remaining 
work over the April/September timeframe. 
These plans are projected based on what oc- 
curred in the first six months, what is re- 
maining to be done and what resources are 
available for the second half of the fiscal 
year. Other agencies may require adjust- 
ments more frequently, depending on the 
projects and stability of the fund sources. 


The planning process is completed when the 
project tasks and the necessary dollars to ac- 
complish those tasks are realistically sched- 
uled across the fiscal year. The planning por- 
tion of the generic cost and performance 
model of Figure 3 is contained in Figure 6. 
After the planning phase, the organization 
proceeds with the implementation phase. 

Implementation Phase 

To implement CAPS, it is necessary to estab- 
lish a structured mechanism for assigning 
work tasks that can be directly associated 
with the “lowest” designated funding source. 
A “work order” provides the authority to ap- 
ply resources, (human, physical and finan- 
cial) to accomplish the tasks required to sup- 
port the project. Each work order must be ac- 
counted for in the Project Cost Plan (Figure 
4). The work order includes more than just 
the tasks to be accomplished. It also includes 
the schedule for completion, deliverables ex- 
pected, the dollar limit and the suborganiza- 
tions or shops involved in the work. 
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Figure 6. Planning Process 


In a customer/contractor arrangement as oc- 
curs within Federal agencies, the contract 
document provides insight into the mecha- 
nisms used to initiate work. For example, in 
the SSC government contractor environ- 
ment, work requests take the form of 
job orders (JOs) for the research contractor; 
facility service requests (FSRs) for the facili- 
ty support contractor; and technical 
work requests (TWRs) for the technical ser- 
vices contractor. The format used by STL is 
the job order as shown in Figure 7. 

Any number of work requests or job orders 
may be required to carry out any one project. 
These are identified by the JO numbering 
system, which is directly related to the pro- 
jects and organization. Some intelligence has 


been built into this system in that the first 
three letters of the job order number are the 
benefitor code (funding source). The fourth 
and fifth numbers are simply the sequence 
number and the sixth and seventh numbers 
are the division number from which the job 
order originates. 

For example, a job number at STL is 
VVD.01.20; where VVD is the benefitor code, 
01 is the sequence number and 20 indicates 
that the job order originated in the HA20 di- 
vision. This numbering system provides a 
mechanism for tracking personnel and 
equipment charges. Personnel time cards 
and labor distribution sheets reflect job order 
charges. These records are the fundamental 
database for the subsequent accumulation of 
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SUPPORT CONTRACTOR JOB ORDER 


DATE 

10/15/90 


CONT CONTROL NO 


MaCIMMHa CMUr 


CONTRACTOR 

LESC 


TASK • JOB ORDER NO 

CS B0-07-01 


ORGINATOR 

Penton 


CONTRACTOR PROJECT NO 

018-CSBO7 


EXTENSION DIVISION NASA CONTRACT NO 

1932 HA00 NASI 3-31 5 


ACCOUNTING CODE NO 


PRESENTATION GRAPHICS 


DESCRIPTION 

Background: Graphic arts services are required to produce materials required tor briefings of general, 
non-project topics. Only such generic, all-STL topic materials should be produced on this job order. Where there 
Is doubt as to the scope ot the requested support, the Technical Monitor, or identified alternate^) should be 
consulted. In many cases, materials must be produced and modified on short notice, thus establishing the need 
lor the capability to respond to 'quick -react ion" requests. 

Tasks: 

1 . Maintain necess.-'Y graphic arts supplies and equipment to support this job order. 

2. Maintain a lie ot reproducible masters of graphics, purging file as requested by technical monitor. 

3. Produce graphic arts in response to requests Iron STL technical monitor. 

4. Coordinate and monitor any graphics work sem to other SSC contractors. 

5. Provide oversight to cost-effective operation ol graphic functions and recommend changes to technical 
monitor as appropriate. Maintain adequate records so that technical montior will have Immediate and full 
Insight In deliverable schedules. Such schedules are to be established when individual graphic arts tasks 
are accepted. 

Schedules: As established when individual tasks are accepted. 

Deliverables: 

1 . Graphic arts products as specified when requested. 

2. Full records of production In weekly activity report. 

3. Other records of production when audits are required. 

Cost of J.O. not to exceed $19,608. 


RESOURCES REQUIRED 


SCHEDULE 



$19,608 


CONTRACTOR COORDINATION f OATE 


SECTION SUPERVISOR 


TECHNICAL SUPERVISOR 


PROGRAM MANAGER 


DISTRIBUTION 





10/1/90 


DATE COMPLETED 


NASA CONCURRENCE/APPROVAL 


TECHNICAL MONITOR 


TECHNICAL MGR'S REP 


GROUP CHIEF 




7/31/91 




Figure 7. Job Order Form 
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project costs. As the work is accomplished, An individual monthly job order report is 

charges (costs) are made to the job order. furnished by the contractor. A sample report 

These charges are reflected in the cost plans from STL is shown below in Figure 8. 
as illustrated in Figure 4. 


RUN DATE 06/28 

JOB ORDER REPORT FOR TOE MONTH OF 
JUNE 


JOB ORDER WBO.01.20 

Ice Detection Prototype 

JOB ORDER TOTAL S 

LABOR 

YEAR TO 

CURRENT 

DATE 

MONTH 


ACTUALS 

ACTUALS 

Hours 

ST Hours 

1836.9 

158.6 

OT Hours 

131.2 

0.0 

Total Hours 

1968.1 

158.6 

MYE 



ST Labor 

1.3 

1.1 

OT Labor 

0.0 

0.0 

Total MYE 

1.4 

1.1 

Cost 

ST Labor 

$41,819 

$ 4.192 

OT Labor 

2,665 

0 

Shift Dili 

0 

0 

OT Premium 

15 

0 

Total Cost 

$44,499 

$ 4.192 

Overhead 

$16,100 

$ 1,551 

Subcontracts 

$ 0 

$ 0 

Equipment 

$ 40 

$ 0 

Material 

$ 2,760 

$ 0 

OTHER DIRECT COST 

Travel 

$ 4,469 

$ 742 

Services/Leases 

2,916 

3 

Relocation 

0 

0 

Use Tax 

188 

124 

Total ODC 

$ 7,573 

$ 869 

Total Cost Before G4A 

$70,972 

$ 6,612 

G4A 

$ 4.258 

$ 397 

Cost of Money 

S 50 

$ 5 

Total Cost Before Fee 

$75,281 

$ 7,013 

Fee 

$ 4,483 

$ 425 

Indirect Cost 

$ 0 

$ 0 

TOTAL COST 

$79,764 

$ 7,438 


OBLIGATIONS 

MATERIAUEQUIPMENT $ 9 927 

TRAVEL $ 0 


Figure 8. Job Order Report 


69 




READINGS IN PROGRAM CONTROL 


Similar to the cost plans, the job order report 
is broken into cost elements and shows the 
monthly data as well as Year-to-Date (YTD) 
data. If a principal investigator or project 
manager has multiple job orders under one 
benefitor, in various suborganizations or 


shops, the activities and tasks can be accom- 
plished as well as costs in the different areas. 

The cumulative costs for the benefitor are 
shown in the contractor’s 533M (Monthly) 
Report, shown below in Figure 9. 


HUN DATE 06/28/ 
BENEFITOR CODE WBO 

benefitor code report for the month of 

June 

SIDS/IDS 

COST PLAN DATE 04/17/91 
COST PLAN TOTAL $ 450,782 

HA20 SPIERING 

LABOR 

YEAR TO 

YEAR TO 

O/U 

VARIANCE 

CURRENT 

CURRENT 

O/U 

DATE 

BUDGET 

DATE 

ACTUALS 

MONTH 

BUDGET 

MONTH 

ACTUALS 

VARIANCE 

Hour* 

ST Hour* 

1683.1 

1836.9 

-46.2 

254.5 

158.6 

-95.0 

OT Houn 

91.2 

131.2 

40.0 

0.0 

0.0 

0.0 

Total Hour* 

1974.3 

1968.1 

-6.2 

254.6 

158.6 

-95.9 

MYE 

ST Labor 

1.4 

1.4 

0.0 

1.8 

1.1 

-0.7 

OT Labor 

0.1 

0.1 

0.0 

0.0 

0.0 

0.0 

Total MYE 

1.5 

1.5 

0.0 

1.8 

1.1 

-0.7 

Coat 

ST Labor 

$39,094 

$41,819 

$ 2.725 

$ 4,636 

$ 4.192 

$ -644 

OT Labor 

1.841 

2.665 

824 

0 

0 

0 

Shift Diff 

0 

0 

0 

0 

0 

0 

OT Pram mm 

12 

15 

3 

0 

0 

0 

Total Coat 

$40,947 

$44,499 

$ 3,552 

$ 4.836 

$ 4,192 

$ -644 

Overhead 

$14,882 

$16,100 

$ 1.218 

$1,789 

$ 1,551 

$ -238 

Subcontracts 

$ 0 

$ 0 

$ 0 

$ 0 

$ 0 

$ 0 

Equipment 

$ 40 

$ 40 

$ 0 

$ 0 

$ 0 

$ 0 

Malarial 

$ 81 

$ 2,760 

5 2,679 

$ 0 

$ 0 

$ 0 

OTHER DIRECT COSTS 

Travel 

$8,164 

$ 4,469 

$ -3,695 

$2,000 

$ 742 

$ -1.258 

A/C Missions 

0 

0 

0 

0 

0 

0 

Ssrvicas/Laasas 

48 

2,916 

2,668 

0 

3 

3 

Relocation 

0 

0 

0 

0 

0 

0 

Use Tax 

10 

188 

178 

0 

124 

124 

Total ODC 

$ 8,223 

$ 7,573 

$ -850 

$ 2,000 

$ 869 

$ -1,131 

Total Cost Before G4 A 

$64,173 

$70,972 

$ 6,800 

$ 8,625 

$ 6.612 

$ -2,013 

G4A 

$ 3,850 

$ 4.258 

$ 408 

$ 518 

$ 397 

$ -121 

Coat ol Money 

$ 46 

$ 50 

$ 5 

$ 6 

$ 5 

$ '1 

Total Cost Before Fee 

$68,069 

$75,281 

5 7.212 

$ 9,149 

$ 7.013 

$ -2,136 

Fee 

$4,130 

$ 4.483 

$ 353 

$ 490 

S 425 

$ -65 

Indirect Coat 

$ 0 

$ 0 

$ 0 

$ 0 

$ 0 

$ 0 

TOTAL COST 

$72,199 

$79,764 

$ 7.565 

$9,839 

$7,438 

$ -2,201 

OBLIGATIONS 

MATERIAL/EQUIPMENT 

TRAVEL 

$ 9,927 
$ 0 







Figure 9. Benefitor <533M) Code Report 
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The 533M report is a requirement levied on 
the contractor by the government. Contrac- 
tors must provide a monthly summary of 
costs for the current month and YTD costs for 
each project (benefitor). This monthly cost 
summary (533) has basically the same ele- 
, ments of costs as the cost plans. The only dif- 
ference is the contractor overhead, G&A and 
fee statements. The current month’s budget, 
current month’s actual cost and variance are 
listed in the right three columns. The YTD 
budget, YTD actual cost and variance are 
listed in the left three columns next to the 
elements of costs. Project managers must un- 
derstand the variance column of this report 
in order to judge performance of the contrac- 
tor. 

At this point in the generic model, the imple- 
mentation phase is complete. This process is 
shown in Figure 10. The subsequent phase is 
the analysis of the data on a monthly basis. 


Analysis Phase 

The analysis phase of the process provides 
the analysis of the actual against the 
planned cost and scheduled milestones data. 
This comparison results in a difference of 
planned versus the actuals called “variance.” 
Analyzing these comparisons results in con- 
clusions relative to project performance. The 
results are then compared in the cost and 
schedules milestone matrix in Figure 11. 

The cost analysis results in a cost underrun, 
on-target or overrun. Schedule milestones 
analysis provides under achievement, on- 
target, or over achievement. The combina- 
tion of these two sets of parameters yields a 
performance indication. 

The purpose for the variance analysis is to 
provide an understanding of trends or impli- 
cations of performance to reduce manage- 
ment surprises and enable early corrections. 



Figure 10. Implementation Phase 
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The matrix in Figure 1 1 provides the combi- 
nation of cost and scheduled milestone ac- 
complishments that translate into perfor- 
mance. Performance is an indicator of pro- 
ductivity reflected in the planning of work 
and its associated dollars, and measurement 
of actual accomplishments and costs. 


COST 

SCHEDULED MILESTONES 

Under 

Achieve 

On 

Target 

Over 

Achieve 

Underrun 

Question- 

able 

Good 

High 

On Target 

Weak 

Good 

High 

Overrun 

Poor 

Weak 

Question- 

able 


Figure 11. Performance Matrix 


The analysis of the performance matrix indi- 
cates the “underrun of cost” row is the best 
performance posture except in the case of un- 
der achievement of milestone accomplish- 
ment. The under achievement of milestones 
accomplishment and underrun of cost is a 
lack of performance based on the dimension 
of time and milestones scheduled. This un- 
derrun condition may indicate potential 
problems or significant cost overruns. The re- 
maining performances in the “cost under- 
run” row go from good to high performance 
when meeting or exceeding the scheduled 
milestones. 

The “on-target cost” row proceeds from weak 
to good and finally to high performance with 
Under achievement, on-target, and over 
achievement of scheduled milestones. The 
“cost overrun” row provides a poor, weak or 
questionable performance when transcend- 
ing from under achievement, on-target and 
over achievement of scheduled milestones. 

The relative degree of high, good, weak, poor 
and questionable performance can be deter- 
mined as a consequence of the detailed ana- 
lysis of cost and scheduled milestone data. 
Examining these elements and questioning 
the variances from project plans provides ex- 


tensive insight as to the absolute magnitude 
of performance. With the actual costs in the 
accomplishment of planned tasks, these costs 
can be examined at the data entry level of 
the personnel time card and labor distribu- 
tion sheet at the job order or work request 
level during detailed analysis or anomaly 
resolution. 

From the initial planning, changes in pro- 
jects can and will occur. However, the initial 
cost plan was constructed from the original 
planned milestones and schedules. While 
maintaining the original plans, the rationale 
and reasonableness of variation between 
what was planned and what actually occurs 
can be assessed and determined. From this 
baseline, the analysis and variance explana- 
tion is understood and may be accepted with- 
out correction to the project. 

A project correction may require redistribu- 
tion of resources, application of additional re- 
sources, resolution of problems, etc. Accep- 
tance without correction requires an under- 
standable explanation of the differences in 
cost expenditures and achievement of miles- 
tones from what was initially planned. 

Understanding that plans of all types are 
subject to change, the initial planning in this 
context is held constant for the first six 
months of the fiscal year. At midyear, the 
cost plans and milestone schedules are re- 
adjusted for the remainder of the fiscal year. 
The project carryover (planned and unplan- 
ned) into the next fiscal year is noted and 
tracked for continuity and used in establish- 
ing a credible plan for the next year. Having 
contingency funds available covers any un- 
certainties of project dollars into the new fis- 
cal year. 

The cost plans, cost actuals and mile- 
stones/schedules, both planned and actual, 
are documented and aggregated at the pro- 
ject level, then to the fund sources level in 
the Financial Reporting Systems (Figure 12). 
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Figure 12. Analysis 


Documentation 

The records of plans and actuals from the de- 
tailed level through the aggregation at the 
division level, directorate level, Center level 
and finally a summary to the Headquarters 
level, constitute documentation of CAPS. All 
data, regardless of the level of summary or 
details, are definable to the lowest level in 
the process. Variance explanations are sum- 
marized and carried through all levels of ag- 
gregation. 

Summary 

CAPS is an automated system used from the 
planning phase through implementation to 
analysis and documentation. Data is avail- 
able or retrievable for analysis of cost versus 
performance anomalies. CAPS provides a 
uniform system across intra- and inter- 
organizational elements. A common system 


is recommended throughout an entire cost or 
profit center. Data can be easily accumulated 
and aggregated into higher levels of tracking 
and reporting of cost and performance. 

For effective program management and con- 
trol to exist, an environment of accountabil- 
ity of organizational elements and individu- 
als must exist. The implication is that the 
level and quality of performance or produc- 
tivity is indicated in the CAPS model and its 
process. Management overview and the 
monthly reporting and analysis provides a 
mechanism for management change, redirec- 
tion or support of the project’s progress. 

The CAPS model provides the necessary “de- 
cision” information and insight to the princi- 
pal investigator/project engineer for a suc- 
cessful project management experience. In 
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fact, CAPS provides all levels of manage- 
ment with the appropriate detailed level of 
data. 

The CAPS model is a disciplined process for 
obtaining required feedback necessary for 
measuring performance on programs and 
projects. It is recommended for cost and per- 
formance tracking for any size or number of 
projects. The results indicate productiv- 
ity/performance and successful project man- 
agement. 

CAPS has been implemented utilizing differ- 
ent software and hardware systems. It is cur- 
rently residing on a PC-based system and an 
institution minicomputer. The concept and 


system are adaptable to any high level data- 
base and project networking software. De- 
pending on the number of projects, in most 
cases CAPS can be handled by PC hardware 
and software. 

The CAPS model provides for planning, im- 
plementation, analysis and tracking of pro- 
jects. Projects utilizing this system at SSC 
range from several thousand dollars a year to 
over a million dollars per year. SSC also uses 
the system for multimillion dollar projects. 
The principal investigator or project engi- 
neer has the responsibility and authority to 
implement the project. CAPS provides a con- 
sistent and uniform system to plan, imple- 
ment, analyze and track a project. 
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Planning and Scheduling for Success 


by Ignacio Manzanera, CCE 

Planning and scheduling should be per- 
formed according to the kind of organization 
and project you have at hand. Selecting the 
ideal planning and scheduling environment 
for your business may give you the competi- 
tive edge. This paper presents the reference 
information required for setting up the most 
convenient planning and scheduling (P&S) 
system and provides useful reference tables 
to assist the user decision-making at critical 
milestone dates. The four sections of the pa- 
per are: 

1. Understanding planning and scheduling 
and their role in the capital projects envi- 
ronment. 

2. Recognizing the importance of early P&S 
and the need to develop your project ac- 
cordingly. We should realize the need to 
measure results, evaluate changes, train 
newcomers, keep historical data, control 
costs, establish financial constraints and 
constantly improve cost and time esti- 
mates. 

3. Respecting human factors in the P&S de- 
velopment cycle, the common denomina- 
tor of consolidated business success. The 
key roles are played by teamwork, profes- 
sionalism and discipline in capital pro- 
jects development. 

4. Planning and scheduling development, 
routines, presentation and the selling of a 
product. 

Understanding Planning and Scheduling 
Some project people think of planning and 
scheduling as yet other management re- 
quirements to be complied with. They do not 
even differentiate between the two functions 
and consequently fail to see their value as 
tools for measuring results, evaluating 
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changes, training new employees, keeping 
historical data organized, controlling costs 
and time and establishing financial needs. 

Planning is an innate human trait that al- 
lows everyone to visualize what has to be 
done and to proceed accordingly. This may be 
easy to do when the project in mind is some- 
thing that does not contain more than 20 ac- 
tivities. Beyond that boundary, most of us 
will need to make a list of all the activities 
and possibly their sequential order to ascer- 
tain the accomplishment of all of them. 

A list may not be enough when the number of 
activities becomes more than 100 and the 
necessary resources have to be accounted for. 
A management procedure, usually accompa- 
nied by a computer system, may be the only 
way to visualize the total work and to com- 
municate it to the rest of the project team. 

Table 1. 

Checklist for Project Planning Basics 

• Construction equipment availability 

• Work site conditions and storage place 

• Construction duration and expected weather 
conditions 

• Performance expected constraints 

• Procurement market status and expected 
trends 

• Labor availability and expected productivity 

• Temporarily needed utilities 

• Local bylaws, studies and interpretation 

The primary planning objectives are con- 
cerned with getting things done within the 
shortest available period of time, minimizing 
cost and risk, and complying with the re- 
quired technical specifications. Table 1 
shows the basic subjects that should be ad- 
dressed by project planners. 
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Table 2 lists the elements needed to achieve 
desired results. 


Table 2. Planning Elements 


• Objective: 

• Program: 

• Budget: 

• Forecast: 

• Policy: 

• Procedures: 

• Standard: 


Goals/target/quotas to be 
accomplished 

Strategy to be followed 

Resources and expenditures 
organized logically 

Projections of what is going to 
happen 

Guidance for decision-making 

Detailed methods for carrying 
out a policy 

Accepted performance level 


Scheduling is a management tool that pro- 
vides time and other resource allocation to 
previously designed plans. Scheduling is one 
of the simplest and least sophisticated tools 
available to project management. With a 
good team effort at the beginning of the job 
and undivided support, it can be a powerful 
project control tool. Table 3 presents a check- 
list for project scheduling basic activities. 

Table 3. 

Checklist for Project Scheduling Basics 

• Set up a work breakdown structure (WBS) 

• Interface the organization breakdown 
structure with the WBS 

• Sequentially organize project activities 

• Run critical path calculations and establish 
project duration 

• Establish a progress measuring system 

• Communicate results, reviews and revisions 


Unfortunately, scheduling is usually ne- 
glected by management because of the level 
of complexity that it typically requires. 


Recognizing the Importance of Early 
Planning and Scheduling 

P&S should start at the project conceptual 
stage, for obvious reasons. Some of them are 
discussed below. 

Maintaining Solid and Consistent Cost 
Estimates. Enforcing P&S from project in- 
ception ensures that everyone within the or- 
ganization will be aware of what is taking 
place. Participation becomes more effective; 
cost estimates are comprehensive and accu- 
rate. 

Designers’ communication with the rest of 
the team becomes a question of simply updat- 
ing plans and schedules and distributing 
them. Cost estimators will have a clear idea 
of the scope of the job, regardless of how 
many changes take place during the develop- 
ment of the project. 

Designers will feel comfortable knowing that 
cost and time estimates are faithful repre- 
sentations of what they are designing. Cross 
checking the estimators’ work is made easy 
by following the work breakdown structure 
and logic sequence of events created by using 
P&S. The real reward comes from having a 
tool that keeps the project team working pur- 
posely for a common objective. 

Instant Evaluation of Changes. There is 
nothing more distressing when working on a 
project than the inability to efficiently evalu- 
ate change in the project design or establish 
a comparison between two alternative 
courses of action. P&S provide effective 
means to know which areas of the project are 
bound to be affected by the decision. Total 
project expected impact in terms of cost and 
time are immediately evident. Again, all pro- 
ject members will be able to visualize the 
proposed situation and make valuable contri- 
butions. 
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Establishing Financial Needs. When cash 
flow and other financial implications for the 
project can be established at an early stage in 
the project, designers have a better chance to 
come up with the right ideas. The cash flow 
usually dictates what can be done and when. 
It would be silly to ignore this fact when de- 
signing and planning a project. P&S provide 
a clear path of expenditures through the ex- 
pected project life. Management has little 
problem understanding the project financial 
needs and matching the required cash flow to 
the plans. 

Measuring Results. Knowing where the 
project stands at all times is essential for pro- 
ductivity, cost control, future commitments, 
coordination and morale. Some people are 
scared of P&S because they think they look 
bad when they do not perform exactly as 
planned. Plans are seldom followed to the let- 
ter; they provide an excellent tool to target 
effort exactly as planned. 

Storing Historical Data. Gathering infor- 
mation for future reference is an endeavor al- 
ways pursued by organizations that are in 
the business and want to continue being part 
of it. P&S supplies a systematic and orga- 
nized tool to keep records of project develop- 
ments. Original plans and schedules and the 
revised ones provide the sequence of events 
that took owner, designer and contractor to 
the final product. Future project planning, 
scheduling, cost estimating and financing of 
similar projects will have a backup reference 
to guide their development. 

Training Newcomers. Through the first 
stages of its life cycle, a project continually 
builds up labor levels. The introduction of 
newcomers to project scope and status could 
be extremely easy if P&S has been running 
from the beginning of the project. Original 
plans, propositions, changes, updates, analy- 
sis and the like will be registered on the 
achieved schedule revisions. Visualization of 
the project status is available at all times. 


Human Factor Analysis 
Bookstores are flooded with books describing 
successful business ventures and the reasons 
behind them. A common denominator among 
them is “teamwork.” A project team is a 
group of persons with different backgrounds 
working for a unique objective, the successful 
completion of a particular project. It is essen- 
tial for all members of a project to work as a 
team, contribute their expertise, share all in- 
formation and maximize resource utilization. 

Owners, designers and constructors may 
have different ideas about their functional 
goals, but when they work together as part of 
a project management team, they must un- 
derstand their jobs as members of this unit. If 
the project team’s objectives statement is 
wrong, everything that follows will be 
wrong, too. Starting with a clear under- 
standing is essential. 

People in project teams usually know their 
job descriptions, their benefits package, and 
their own job objectives. But their ideas 
about the project team’s mission are fre- 
quently vague. As a group, project team 
members may never have articulated their 
objectives to one another. P&S makes every- 
one in the project team realize the impor- 
tance of: 

• Understanding their job as part of a team. 

• Having a clear idea of the project objec- 
tives and how the team, as a group, is pur- 
suing them. 

• Identifying critical milestones to be 
reached and the expected contribution 
from the team members. 

P&S fosters team member communication 
and interfaces, clarifies assumptions about 
leadership, encourages competition, guides 
reward systems and enhances probabilities 
of success. Other team-related skills such as 
flexibility, patience, ability to check out as- 
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sumptions, willingness to listen to others, 
curiosity and versatility are encouraged by 
P&S among project team members. P&S edu- 
cates project managers to trust the ability of 
effective teams to outperform individuals. 

Planning and Scheduling Development 

The key issue in selecting a P&S working 
package is simplicity. It must be kept in 
mind that for a P&S program to work, it 
must be clearly understood by all. If it is con- 
fusing, it will not be followed; and if it is not 
followed, people will not make the necessary 
effort to make it accurate, and consequently, 
it will become useless. 

Reports must be kept short so anybody can 
read them, and graphics must be abundant 
so people bear in mind what is going on. 
Needless to say, accuracy should be consis- 
tent so that everybody in the project is guid- 
ed by the P&S program. Different project fo- 
rums should use different levels of detail. It 
would be a useless exercise to try to expose 
management to all schedule calculations. 
This belongs to the planning and scheduling 
engineers. By the same token, it would be ri- 
diculous to think that engineers can manage 
all aspects of a job using a simple bar chart. 

P&S Needs at Project Conceptual Stage. 
The main activity for P&S at project concep- 
tual stage is identification of the scope of the 
job. (Table 4). It means that every input by 
the conceptual engineers must be translated 

Table 4. Checklist for P&S Activity 
at the Conceptual Stage 

• Gain total project management team 
commitment on the P&S program. 

• Establish update periods, feedback deadlines, 
level of detail and distribution list. 

• Decide on computer applications to use. 

• Identify scope of work. 

• Generate a work breakdown structure. 

• Introduce first P&S program draft and clearly 
establish its expectations. 


to activities to be planned and scheduled. 
Once all activities have been identified, the 
work breakdown structure must be estab- 
lished to keep the work systematically orga- 
nized. Finally, a master milestone schedule 
must be developed to help the team visualize 
the total project. 

P&S at Project Proposal Stage. Project 
proposal development brings more detail to 
the plans, and it should be inserted into the 
incipient P&S program (Table 5). 

Table 5. Checklist for P&S Activity 
at the Project Proposal Stage 

• Set up the OBS. 

• Define the OBS and WBS interface. 

• Generate project code of accounts. 

• Create work packages. 

• Introduce a project proposal progress 
measuring system. 

• Develop the first network schedule and 
update it periodically according to new 
information generated by the project 
proposal progress. 

• Resource load the plan and schedule and 
identify long lead time material and 
equipment. 


An organization breakdown structure (OBS) 
should be set up to establish management re- 
sponsibility. Then, by interfacing the OBS 
with the WBS, working packages are set up. 
A breakdown of the work packages into cost 
items will provide the basis to develop the 
schedule network that will guide the project. 
An improved P&S program will help develop 
a better understanding of the scope of the job 
and generate more accurate cost/time esti- 
mates. Long lead time material required for 
the project must be clearly identified and 
scheduled according to the network plan. 

P&S During Definitive Design. When pro- 
ject expenditures have been approved and de- 
finitive design is awarded, all the P&S effort 
developed during the previous project life 
phases will start paying off. Having a well 
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identified and organized schedule network 
with the solid foundations of WBS, OBS and 
work packages, planning and scheduling the 
definitive design phase will not be a problem. 
Table 6 presents a checklist for P&S efforts 
during the design phase. 

Table 6. Checklist for P&S Activity 
at the Design Stage 

• Compare inhouse schedules against awarded 
contractor schedules and revise accordingly. 

• Set up a reliable project progress measuring 
system based on the approved working 
schedule. 

• Revise the company's cost and schedule 
estimates as the design progresses to show 
the detail is continuously introduced and its 
corresponding impact on the plan. 

• Keep a vivid interest on project procurement 
requirements and their impact on the job. 


The number of drawings per cost item can be 
forecasted and labor levels established and 
compared against bidders’ offers. A definitive 
design contractor's working schedule can be 
reviewed and approved accordingly and a 
progress measuring system implemented. 
The project’s overall P&S should be constant- 
ly revised according to authorized-for-con- 
structi on -drawing production. Time and cost 
estimates are adjusted against the latest ma- 
terials and other resources takeoff lists, and 
P&S will provide a clear organization for ev- 
erybody to visualize project status. Special 
attention must be given to planning and 
scheduling requisitions and purchase orders 
for long lead time material and equipment at 
this stage. 

P&S During Construction. This final 
phase of the project will receive most of the 
benefits of the early P&S program. At the bid 
job explanation meeting there will be an in- 
house generated schedule network to help 
bidders understand what they are about to 
enter. It is an excellent tool for getting com- 
petitive bids and thereby cost reductions. 


The previously developed cost items and 
work packages will provide a solid structure 
for bidders’ quotation costing and avoidance 
of dear mistakes. Table 7 shows a checklist 
for P&S development during construction. 

Table 7. Checklist for P&S Activity 
at the Construction Stage 

• Compare inhouse schedules against awarded 
contractor schedules and revise accordingly. 

• Set up a reliable project progress measuring 
system based on the approved working 
schedule. 

• Update schedule resources allocation to 
ascertain that it matches the scope of the job 
and definitive cost estimate. 

• Establish a project change processing 
procedure with emphasis of the approval 
cycle duration. 

• Institute performance indicators to be 
utilized and clarify their interpretation. 

• Review and revise activities working crew 
sizes and their qualification requirements. 

• Establish early warning systems for cost and 
schedule slippages. 

• Establish a direct line of communication 
between cost estimating, scheduling, material 
procurement and surveying, and make all of 
them responsible for the project outcome. 


When bidders submit their proposals, their 
construction schedules can be cross-checked 
against the company’s previously developed 
schedule for accuracy and scope compliance. 

Conclusion 

Planning and scheduling programs are excel- 
lent management tools when properly intro- 
duced to the project management team and 
regularly maintained. Communications, cre- 
ativity, flexibility and accuracy are substan- 
tially improved by following a simple set of 
rules. A planning and scheduling program 
will work for you if you believe in it, make 
others in your project team realize its bene- 
fits, and above all, make it an extension of 
your project cost control philosophy. 
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Scheduling: A Guide for Program Managers 


Defense Systems Management College 

For 4,500 years after the building of the 
great pyramid at Giza, nothing surfaced as a 
better way to develop a schedule than that 
used for the pyramid. Then, early in this cen- 
tury, Henry L. Gantt, a pioneer in the field of 
scientific management, unveiled his bar 
chart technique. From that time forward, 
program planning and scheduling have con- 
sisted of a list of activities with start and 
completion dates. 

Gantt’s “daily balance chart” was a signifi- 
cant breakthrough. Suddenly, you could see 
at a glance the overall program schedule and 
the start and stop times of the program’s in- 
dividual components (Figure 1). A Gantt 
chart can be superimposed with ease on a cal- 
endar. Then, by shading in each bar as 
progress is made, a manager can easily mea- 
sure actual progress against the schedule. 



Now 

TIME/MONTHS 


■ £ rtu31 □ Scheduled ^ Milestones 
Progress Activity 

Figure 1 . Gantt (Bar) Chart 


For 50 years, bar charting was the best way 
to schedule activities. There were good rea- 
sons for it. Bar charts communicate, are easy 
to prepare and use, show key activities with 
specific start and completion times (sched- 
uled and actual), relate schedules to calendar 
dates, and display days or weeks from pro- 
gram start to completion. 

Figure 1 shows that milestones may be added 
to bar charts to display significant events. In 
fact, it may be appropriate to show a number 
of milestones associated with a single bar. 
Because they communicate so well and so 
quickly, bar charts are still used to plan and 
monitor progress against the plan. Upper 
management, in particular, appreciates this 
capability. 

A shortcoming of bar charts is the limited in- 
formation they portray. Dependency and oth- 
er interrelationships among activities are 
difficult to display because bar charts handle 
a limited degree of complexity. Figure 2 
shows how a bar chart can provide a clear, 
but limited, picture of dependencies and 
progress. The bar chart can present a history 
of changes and rescheduling occurring on a 
program; however, this is done more fre- 
quently on milestone charts, which will be 
discussed later. 

As a scheduling tool, the bar chart is simple, 
communicates well, and displays calendar 
and significant program dates. Because of its 
simplicity and ease of interpretation, the bar 
chart is a particularly good tool for communi- 
cating important information to upper man- 
agement; however, it is limited in the degree 
of detail and the interrelationships it can 
portray. 
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Figure 2. Gantt Chart Showing Dependency 


Table 1 summarizes the strengths and weak- 
nesses of Gantt charts. The weaknesses ap- 
ply to planning, estimating and reporting as 
well as to bar charts. 

Milestone Scheduling 

Milestone scheduling is a popular technique 


being used in Department of Defense (DoD) 
program management offices. The milestone 
chart is probably the most commonly used 
chart at the Air Force Systems Command, 
Electronic Systems Division (ESD), and 
many other DoD organizations. The tech- 
nique is relatively simple. Milestone charts- 


Tablel. Gantt Technique: Strengths and Weaknesses 



CRITERIA 

STRENGTHS 

WEAKNESSES 

1. 

Accuracy 

Good in repetitive work. Time 
estimates are likely to be good 
and production is easy to count. 

In non-repetitive work, accuracy of estimate 
of task completion percentage is subject 
to error. 

2. 

Reliability 

Simplicity of technique helps 
program manager to set up 
a consistent progress reporting 
system. 

In repetitive work, production can be 
"doctored." Large non-repetitive programs 
involve many different progress estimators, 
which tends to affect consistency. 

3. 

Simplicity 

Easy to understand, accept 
and implement. 

Requires good time estimates, or standards, 
which are not simple to develop. 

4. 

Universality 
of program 
coverage 

Effective at work-center levels. 
Cover a specific phase of program 
life cycle well. 

Not effective for a large, complex 
program unless program is 
computer-based. 

5. 

Decision 

analysis 

N/A 

No capability to stimulate 
alternatives. 

6. 

Forecasting 

Clearly shows ability to meet 
schedules in repetitive work. 

Does not show ability to meet schedule if 
many interrelated tasks are involved. 

7. 

Updating 

Easy to update if program 
is static. 

N/A 

8. 

Flexibility 

N/A 

Much chart reconstruction needed to show 
program changes. 

9. 

Cost 

Data gathering and display are 
relatively inexpensive. 

Frequent program changes cause costly 
redrafting of charts. 
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are event oriented, bar charts are activity 
oriented. For a particular program, a set of 
key events, or milestones, is selected. A 
milestone is a scheduled event that will occur 
when a particular activity is started or com- 
pleted. Milestones are selected from the pro- 
gram management plan. By reviewing the 
status of key milestones, one can assess 
quickly the overall program status. 

Although milestone charts can present more 
information than bar charts, they share one 
important drawback: they invite surprises. A 
surprise can occur when the number of dis- 
played milestones is too limited or when in- 
terdependencies are not portrayed. The re- 
sult may be that the program manager does 
not know the status of a key event until it oc- 
curs, or until it fails to occur when scheduled. 
A well conceived milestone status report can 
provide early warning of a potential problem, 
and early problem recognition is a key to suc- 
cessful program management. 

The milestone scheduling technique uses a 
symbology consisting of arrows and dia- 
monds, or similar designators, to show origi- 
nally planned event dates and the changed 
dates. Figure 3 shows the symbols and their 
meanings used by the Air Force at ESD. Any 
symbol can be used; mechanics are not as im- 
portant as principles. 

Arrows are used to show rescheduled events; 
diamonds indicate the originally planned 
schedule. As a result, the milestone schedule 
allows us to improve on the Gantt chart by 
retaining the baseline dates, while incorpo- 
rating changes in planned future events. Fig- 
ure 4 is an example of a milestone chart. 

The milestone chart records the manager’s 
assessment. For example, a manager might- 
reasonably predict that a one-month slip in 
the start of software development will prob- 
ably result in a several-month slip in com- 


pleting the engineering development phase. 
The milestone chart does not provide the as- 
sessment - the manager’s experience does. 
This is the key to understanding the use of 
milestone charts. Unless the activity and in- 
terrelationships of milestones are under- 
stood, the chart tells only what has happened 
and what is yet to happen. However, by cou- 
pling historical information with the man- 
ager’s experience and knowledge, more accu- 
rate predictions can be made. 

The milestone scheduling technique shows 
what is scheduled, what has happened and 
changes in plans. The technique is not as 
useful for forecasting schedule changes as 
are the network and line-of-balance tech- 
niques discussed later. 

Like the bar chart, the milestone chart is an 
effective method of communication. The sym- 
bology is relatively standard and simple to 
use. The chart presents actual progress 
against a baseline plan and displays changes 
in plans. The mechanics of constructing a 
milestone chart are relatively easy. Many de- 
fense contractors use milestone charts exten- 
sively in DoD program management. 

As with simple bar charts, a major weakness 
of milestone charts is their inability to show 
interdependencies and interaction among ac- 
tivities. A potential problem can result if a 
program manager focuses on a relatively 
simple milestone format. He may lose sight 
of the complexity of the relationships among 
various program tasks. 

Although milestone charts are used on com- 
plex programs, they are usually the product 
of network analysis. Milestone chart prep- 
aration is relatively simple, but developing 
and analyzing the information going into the 
charts can be time consuming. A controlled 
flow of accurate, timely and appropriate in- 
formation is important. 
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Standard symbols have been adapted for Air Force 
milestone schedules. The most common symbols 
used and their meanings are shown below. 


Basic Symbol 


Meaning 

Schedule completion 

Actual completion 

Previous scheduled 
completion— still in future 

Previous scheduled 
completion— date passed 


Representative 

Uses 


Meaning 

Anticipated slip- 
rescheduled completion 

Actual slip- 
rescheduled completion 

Actual slip— actual completion 

Actual completion ahead of 
schedule 

Time span action 

Progress along time span 

Continuous action 
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Figure 4. Milestone Chart 
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Milestone charts represent a simple and ef- 
fective means to display actual versus 
planned progress of a program, and to show 
schedule changes that have occurred. These 
charts emphasize start and completion dates, 
rather than the activities that take place be- 
tween these dates. Milestone charts display 
very limited dependency information and 
they may present program status in a decep- 
tively simple manner. 

Network Scheduling 

Shortcomings of bar and milestone charts 
gave rise in the 1950s to network scheduling. 
The network techniques provided a may to 
graphically display information for program 
managers that could not be presented with 
bars or milestones. 

First, a program, is separated into activities. 
Each activity is based on a particular under- 
taking and each is defined by a distinct start 
and completion point. Network scheduling 
provides a method for finding the longest 
time-consuming path. This gives the man- 
ager two important tools. First, the project 
manager is able to more accurately estimate 
the total time from program start to comple- 
tion. Second, by being able to identify items 
on the critical (or longest) path as opposed to 
tasks less critical, the project manager is 
able to analyze problems as they arise. 

Program Evaluation and Review 
Technique 

The Program Evaluation and Review Tech- 
nique (PERT) was developed in 1957 under 
the sponsorship of the U.S. Navy Special Pro- 
jects Office. The Navy wanted PERT as a 
management tool for scheduling and control- 
ling the Polaris missile program, a program 
which involved 250 prime contractors and 
more than 9,000 subcontractors. The project 
manager had to keep track of hundreds of 
thousands of tasks. 

PERT helps answer such questions as: 


• When is each segment of the program 
scheduled to begin and end? 

• Considering all of the program segments, 
which segments must be finished on time- 
to avoid missing the scheduled comple- 
tion date? 

• Can resources be shifted to critical parts 
of the program (those that must be com- 
pleted on time) from non-critical parts 
(those that can be delayed) without affect- 
ing the overall scheduled completion date 
for the program? 

• Among the myriad program tasks, where 
should management efforts be concen- 
trated at any particular time? 

Since most activities in a PERT network 
take a long time to accomplish, time is usual- 
ly expressed in days or weeks. The expected 
time for an activity is often described by a 
probability distribution rather than a single 
estimate because of the uncertainty associat- 
ed with programs that have not been done 
the same way before. The characteristics of 
the distribution used to express the variation 
in time are: 

• A small probability of reaching the most 
optimistic time (shortest time), time a. 

• A small probability of reaching the most 
pessimistic time (longest time), time b. 

• The one most likely time which would fall 
between the two extremes above, time m. 

• The ability to measure uncertainty in 
estimating. 

Because it has all four attributes, the beta 
distribution was chosen for determining the 
expected time. Figure 5 shows a beta distri- 
bution with the time designations under the 
curve. 
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Figure 5. Beta Distribution with Symbols 
Indicating Time Estimates 

The three time estimates shown may be com- 
bined into a single workable time value. By 
using the following weighted average formu- 
la, the expected time for an activity can be 
found: 

t = a -f 4m + b 
6 

where t is the expected time. According to 
MacCrimmon and Ryavec, the error in the 
answer using this formula is small enough to 
make it satisfactory to use in most cases. 

According to Hugh McCullough, former Po- 
laris project business manager, PERT had a 
disciplinary effect. The Polaris project had a 
20,000-event network and the application of 
PERT is credited with saving two years in 
bringing the Polaris missile submarine to 
combat readiness. 

As PERTs popularity grew, consulting firms 
specializing in network scheduling sprang up 
overnight. The DoD established a PERT Op- 
eration and Training Center (POTC, nick- 
named “Potsie") in Washington, DC. During 
the next few years, PERT became widely 
used throughout DoD systems acquisition 
programs. 

A few years later, the use of PERT declined 
sharply, and by the 1970s, it was rarely em- 


ployed in defense systems programs. Why did 
PERT go through such a rapid rise and 
abrupt decline? In essence, the predictable 
happened. When PERT was combined with 
cost data or other non-scheduling aspects of 
program management, it became cumber- 
some. Eventually, use of such an embellished 
technique resulted in the tail wagging the 
management dog. The DoD program manag- 
ers and defense contractors spent immense 
amounts of time collecting and entering de- 
tailed data. Soon, the cost of maintaining 
PERT systems far outweighed the benefits 
they offered the program manager. 

An article published in the DSMC magazine 
Program Manager reveals how little network 
scheduling is being used by DoD acquisition 
program managers. Seventy percent of the 
major defense programs surveyed do not use 
a network scheduling system. However, the 
remaining 30 percent employ a network 
technique. (Ingalls) 

DoD and the defense industry returned to 
simpler techniques like milestone and bar 
charts, probably an overreaction. The private 
sector continues to make good use of network 
scheduling in varied efforts like new product 
development, construction, and major main- 
tenance activities. This resurgence is due in 
part to the development of PERT and other 
networking software programs that run on 
microcomputers. 

In spite of misuses that have occurred in 
PERT applications, the technique enables 
the manager to visualize the entire program, 
see interrelationships and dependencies, and 
recognize when delays are acceptable. Thus, 
the manager is better able to assess problems 
as the program evolves. In order to apply 
PERT and similar networking techniques, it 
is important that certain conditions exist: 

1. The program must consist of clearly de- 
fined activities, each with identifiable 
start and completion points. 
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2. The sequence and interrelationships of 
activities must be determined. 

3. When all individual activities are com- 
pleted, the program is completed. 

Many program-oriented industries, like 
aerospace, construction and shipbuilding, 
meet these criteria and use a network sched- 
uling technique. Many defense system pro- 
grams also meet these criteria. 

Critical Path Method 
Like PERT, the Critical Path Method (CPM) 
is composed of three phases: planning, sched- 
uling and controlling. This technique, devel- 
oped in 1957 by J.E. Kelly of Remington- 
Rand and M.R. Walker of DuPont to aid in 
scheduling maintenance shutdowns in 
chemical processing plants, is essentially ac- 
tivity oriented; PERT is event oriented. CPM 
has enjoyed more use among network tech- 
niques than any other technique. 

CPM brings the concept of cost more promi- 
nently into the scheduling and control pro- 
cess than PERT does. When time can be esti- 
mated closely and when labor and material 
costs can be calculated early in a program, 
the CPM technique is superior to PERT. 
When there is much uncertainty and when 
control over time outweighs control over 
costs, PERT is a better technique to use. The 
basic networking principles in PERT and 
CPM are similar. 

In a common version of CPM, two time and 
cost estimates are given for each activity in 
the network. These are the normal estimate 
and the crash estimate (see Figure 6). The 
normal time estimate approximates the most 
likely time estimate in PERT. The normal 
cost is the cost associated with finishing the 
program in the normal time. The crash time 
estimate is the time that will be required to 
finish an activity if a special effort is made to 
reduce program time to a minimum. The 
crash cost is the cost associated with per- 


forming the effort rapidly, in order to mini- 
mize the time to completion. 



Figure 6. CPM Time-Cost Curve 

Developing a Network 
Although CPM and PERT are conceptually 
similar, their symbols and charting tech- 
niques vary. The PERT historically has used 
probability techniques while, CPM generally 
does not. The following procedure applies to 
both CPM and PERT. 

1. Identify all individual tasks comprising 
the program. 

2. Determine the expected time to complete 
each activity. 

3. Determine precedence and interrelation- 
ships of the activities. 

4. Develop a network diagram presenting 
these activities in proper sequence and re- 
flecting any dependency relationships. 
Activities are indicated by lines; events or 
milestones are indicated by circles. De- 
pendency or sequencing relationships 
among activities on separate paths can be 
shown by dotted lines (dummy activities). 

5. Compute and annotate the cumulative 
time required to reach each milestone 
along the paths, which will indicate earli- 
est time work can start on the next activ- 
ity. The final number will indicate the to- 
tal time required to complete a path. 
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6. Identify the critical path. This is the se- 
quence of events, or route, taking the 
longest time to complete. 

7. Starting at the program completion 
milestone on the right side of the dia- 
gram, begin working backward and com- 
pute the latest time an activity can start 
without delaying the overall program. 
For example, if the total program takes 
40 weeks and the last activity requires 
five weeks, the final activity cannot begin 
later than week 35. The difference be- 
tween the earliest start time and the 
latest time before each activity is the 
slack time, or float. The critical path con- 
tains no slack time. 

Figure 7 shows a simple network diagram for 
a computer installation program. This net- 
work diagram shows the total program will 
require 20 days to complete. The critical path 


is F-G and any delay on this path will delay 
final completion of program. However, delay 
of one day can occur along path C-D-E, a de- 
lay of five days can occur along path A-B-E, 
and the final completion date would not be 
extended. 

Critical path programs may be either activ- 
ity oriented or event oriented. This means 
that the input and output data are associated 
with either activities or events. The distinc- 
tion between the two is not a substantive one 
with respect to computational practices. 

Although it has not been done in the above 
example involving the installation of a com- 
puter, many CPM programs and a few PERT 
programs require that events (circles on the 
diagram) be numbered in ascending order. 
This inhibits the flexibility of the network 
and causes event-numbering bookkeeping 
problems, so it is not always done. 



Program Activity Program 

Start a - Build raised floor Complete 

B - Build air conditioning vents 
C - Bring special power source to computer room 
D - Install wiring and connect to power source 
E - Install air conditioning 
F - Await delivery of computer 
G - Install computer 


Figure 7. Network Diagram for Computer Installation Program 
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Converting an Ugly Duckling to a Swan 
Although the traditional CPM technique 
provides useful visibility and clarity about a 
program, it has shortcomings in that it is dif- 
ficult to draw the chart to match a time or 
calendar scale. Although the critical path 
and slack times can be computed easily, they 
are not readily apparent. Also, this tech- 
nique does not display progress to date. Con- 
sequently, a simpler technique, sometimes 
called the Swan Network, is useful. 

Let’s take an Ugly Duckling Network (Fig- 
ure 8) and turn it into a Swan Network (Fig- 
ure 9). Letters in Figure 8 represent activi- 
ties between the start (S) and completion (C) 
points. Numbers indicate weeks required for 
each activity. In Figure 9, activity A is repre- 
sented by a horizontal bar four weeks long. 
Constraints are represented by vertical lines 
or “fences;” for example, the fence after B 
means B must be completed before E and F 
can begin (the same as in Figure 8). The re- 
sult is shown in Figure 9. 


What does the Swan network show? 

1. The critical path. Time constraints do not 
have to be calculated. There are, in fact, 
two critical paths, B-F-I and C-J-K, 
which are critical because each has a con- 
tinuous series of activities. There is no 
slack in either path. Also, the figure has a 
time scale, which adds greatly to the 
meaning of the chart. 

2. Weeks from start. Scales for “calendar 
weeks,” and “weeks to completion” can be 
added. In Figure 9, the program is sched- 
uled for completion after 14 weeks. 

3. Where there is slack in the schedule and 
the extent of that slack. For example, 
there are only two weeks of slack in the 
A-D-H path. If B-F-I and C-J-K were 
shortened by more than two weeks, A-D- 
H would become a critical path. This 
changing of two critical paths is impor- 
tant when conducting “what if’ exercises. 



Figure 8. The Ugly Duckling 
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The high visibility offered by the Swan net- 
work does the following: 

• Communicates. 

• Motivates. If the level of detail is suffi- 
cient, everyone associated with a program 
activity can see how they affect the sched- 
ule, and vice versa. 

• Gets top-level attention. 

• Makes omissions and errors easier to de- 
tect; for example, one company discovered 
that by using the Swan network, two test 
activities on the critical path had been 
omitted. (This was not apparent in the 
Ugly Duckling network.) 

• Shows early start, early finish, late start 
and late finish. 

• Avoids reams of printouts, provided (but 
not used) for the Ugly Duckling. 


The Swan network can be developed in sever- 
al ways; it can be translated from another 
network, as shown in the preceding example; 
it can be developed from a listing of the pre- 
ceding and following events or activities, as 
in the network scheduling problem that fol- 
lows; it can be developed from scratch, with 
the sequencing and time estimating required 
in originating any network; and it can be de- 
veloped from milestones. 

A “fence” in the Swan network is usually a 
milestone like a review or a major event, re- 
gardless of how the network is developed. 

Actual progress can be shown in the same 
way as on a Gantt chart. Shading on each bar 
indicates progress made. A vertical “now” 
line shows whether activities are on, ahead 
of, or behind schedule, and by how much. 

Now, let’s go through an exercise involving 
network scheduling. Take time to work the 
problem shown on the following pages. 
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Network Scheduling Problem 

Assume you are program manager. Your ob- 
jective is to schedule the activities on your 
program so that one lot of missiles will be as- 
sembled and shipped to the test site within 
56 days and at the least cost. Use any tech- 
nique with which you feel comfortable. If 
you’re not comfortable with a particular 
technique, use the Swan network. Proceed in 
the following manner, using Tables 2 and 3 
provided. 


Using lined tablet paper, lay out the normal 
schedule. This will show the critical path and 
total number of days required. Identify the 
initial critical path (number of days). Using 
Table 3, select the final critical path and re- 
lated costs that will ensure the completion of 
the program in 56 days and at the least cost. 

It will probably take about 20 minutes for 
you to determine the solution. The cost for a 
56-day program will be in excess of $778,000. 


Table 2. Activities, Dependencies , Times and Costs 


ACTIVITY** 

ACTIVITY 

DEPENDENCY 

TIME 

(WORK DAYS) 

NORMAL COST 
($000) 

1-2 Fab. Initial Guidance 
Assemblies 

None 

12 

60 

1-3 Controls Fabrication 1 * 

None 

24 

96 

1 -4 Rocket Motor Fabrication 

None 

28 

105 

1-5 Process Warheads (GFE) 

None 

16 

37.5 

2-6 Additional Guidance 
Assemblies 

1-2 

20 

90 

2-3 Guidance Checkout and 
Sub-Assemblies 

1-2 

16 

120 

3-5 G&C Sub-Assemblies 

1-3, 2-3 

8 

70 

4-5 Machine Rocket Motors 

1-4 

12 

30 

5-6 Missile Assembly 

3-5, 4-5, 1-5 

6 

37.5 

6-7 Test 

2-6, 5-6 

10 

62.5 

7-8 Ship to Test Site 

6-7 

8 

30 


Note: a. Table 3 contains "crash" data. 

b. Work on controls fabrication cannot start until after day 2 due to limited resources 
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Table 3. Activity Time/Cost Relationships 


Activity 1-2 

Activity 1-3 

Activity 1 -4 

Time Cost 

Time 

Cost 

Time 

Cost 

10 62.5 

22 

105 

26 

110 


20 

117 

24 

115 


18 

127 

20 

142 

Activity 1-5 

Activity 2-6 

Activity 2-3 

Time Cost 

Time 

Cost 

Time 

Cost 

14 60 

18 

97.5 

14 

132.5 

12 67.5 

16 

110 

12 

150 


Activity 3-5 

Activity 4-5 

Activity 5-6 

Time 

Cost 

Time 

Cost 

Time 

Cost 

★ 

* 

10 

35 

4 

60 



8 45 

Activity 6-7 

Activity 7-8 


Time Cost Time Cost 

* * 6 60 


Note: Crash time is in work days and cost is in thousands of dollars. 

Crash costs include normal schedule costs. For example, the Activity 1-2 crash cost ($62. 5K) 
includes the normal schedule cost of $60K. 

The activity marked * cannot be "crashed." 
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Network Scheduling When Resources 
Are Limited 

In the previous discussion, the assumption 
was that a new activity could start as soon as 
preceding activities were completed, because 
sufficient resources were available to per- 
form the work. In practice, however, re- 
sources to proceed are not always available. 

Let’s look at an example to illustrate how 
this network differs in format from previous 
networks. First, it uses curved lines for ac- 
tivities. This eliminates zero-time activities. 
Second, it identifies each activity in three 
ways: by a letter (A), (B), (C), etc.; by esti- 
mated duration of activity (in weeks); and by 
number of people available to work on the ac- 
tivity based on the manager’s estimate at the 
time the network is prepared (see Figure 10). 

The network in Figure 10 can be shown in 
another manner (see Figure 11). In this net- 
work, each activity is plotted on the schedule 
graph with a horizontal time scale. The dura- 
tion of each activity is represented by the 
length of that activity’s line. The description 
of each activity represents its letter designa- 
tion and number of people assigned to that 
activity at the time indicated (size of work 
group). The row across the bottom shows to- 
tal people scheduled to work each week. In 
this example, 5 to 15 people will be needed, 
depending on the week being scheduled. 

Now, let’s suppose only nine people are avail- 
able to work this nine- week period. What al- 
ternatives do we have? We can produce a per- 
sonnel loading chart by plotting the number 
of people scheduled to work in any week 
against time (Figure 12). Then, if we know 
that only nine people are available, we can 
see we will not have sufficient workers dur- 
ing the first, fourth and fifth weeks. We will 
have sufficient workers to perform the work 
scheduled during the second and sixth week. 


During the third, seventh, eighth and ninth 
weeks, we will have a surplus of workers for 
the work scheduled. The task becomes one of 
rearranging the schedule so that the peaks 
and valleys are evened out without schedul- 
ing more work than nine people can do. It 
may not be possible to rearrange the network 
and still finish the program on time, Under 
present circumstances, there will not be 
enough workers to complete the first week’s 
scheduled work on time. 

The scheduling problem we are considering 
can be solved quickly by hand; however, 
when there are many activities, it becomes 
very difficult to find the optimum answer, 
even with a computer. A heuristic program 
should be used to solve this kind of problem. 
The heuristic rule is a rule of thumb that 
works; therefore, collection of rules of thumb 
is usually known as a heuristic program. 

In our example, the heuristic approach is one 
of finding activities having the most slack 
and attempting to delay them as long as pos- 
sible without delaying completion of the en- 
tire program. We can delay the start of activ- 
ity (C) for two weeks and activities (A) and 
(B) can begin simultaneously without ex- 
ceeding the limit of nine workers. Continu- 
ing to apply this approach, the revised sched- 
ule could look like that shown in Figure 13. 

When an activity is delayed to improve the 
schedule, the time which it is delayed is usu- 
ally shown by a dotted line. At the end of the 
third week in our example, we had an oppor- 
tunity to delay activities (D), (G) and (I). We 
chose to delay (D) and (H) one week and (I) 
four weeks. Although our example is simple, 
it is not possible to achieve a perfectly bal- 
anced schedule. Given the complexities of the 
program on which these techniques are often 
used, most managers would be happy to 
achieve the success we did in this example. 
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Figure 10. Network Illustrating Problem When Resources Are Limited 



Figure 11. Limited Resource Network Plotted on Schedule Graph 
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Figure 12. Personnel Loading Chart for Schedule Graph 



Time (Weeks) 


Figure 13. Revised Schedule Graph 
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Multi-program Considerations 
In his dissertation at Purdue University in 
1964, J.H. Mize offered a method for multi- 
method control. He developed a non-iterative 
heuristic model that schedules activities for 
several operating facilities of a multi- 
program organization when the objective is 
to minimize due date slippage. The outputs 
from program critical path analyses become 
the inputs to the schedule. Mize took into ac- 
count the dynamic relationships of activities 
to activities and program to program when 
conflicts arise. The method offered by Mize is 
applicable generally to any program involv- 
ing more than one program competing for the 
same limited resources. 

In 1968, L.G. Fendley developed a system 
based on the concept of assigning the due 
dates to incoming programs and then se- 
quencing activities of the programs toward 
meeting the due-date. He used the heuristic 
approach to solve the scheduling problem. 
Fendley concluded that giving priority to the 


activity with minimum slack-from-due date 
(his MSF rule) resulted in the best perfor- 
mance. He used the MSF rule to set realistic 
due dates by determining the amount of slip- 
page that must occur to perform all programs 
with fixed resources. 

In 1970, Mize and L.F. Jordan applied a sim- 
ulation technique to the scheduling of multi- 
engineering programs. They discovered that 
a rule based upon a combination of process- 
ing time and due date yielded good results. 

All networking concepts can be applied to the 
scheduling of several programs jointly ad- 
ministered by a single organization. For ex- 
ample, consider the three programs shown in 
Figure 14. In this example, Program A must 
be completed before Program B can start. 
Program C and Program D may begin and be 
completed any time between week 1 and 
week 6, respectively. Thus, the dotted lines 
in Figure 14 indicate dummy dependencies 
only. They serve to indicate the time span 
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available for all four programs. Duration 
times could be placed on these dummies to 
achieve early start and late finish program 
dates, if they exist. The program floats im- 
plied by these dummy jobs can be used in the 
same way that dummy jobs are used in 
single-program networks. 

For example, suppose the same resources are 
used on Programs B and C. Furthermore, 
suppose these resource requirements exceed 
the availabilities because of the simulta- 
neous demands by Programs B and C. Figure 
14 shows that the start of Program C can be 
delayed until week 4, while the resources are 
fully employed on Program B. After Program 
B is completed, the resources can be released 
for use on Program C. Alternatively, both 
programs can use the resources at a reduced 
rate, and both programs will then float out 
(as long as they do not float beyond week 6.) 
Whole programs may be cost-expedited. 
Thus, multi-program networking techniques 
are completely analogous to single-program 
networking techniques. 

There is, however, one new aspect in multi - 
program scheduling: program priorities. 
Suppose that Program C (Figure 14) is 
deemed to be the most important program 
and management wishes to have it start be- 
fore any other program. In the Resource Allo- 
cation and Multi-Project Scheduling 
(RAMPS) computer algorithm developed in 
1963 at C-E-I-R, Inc. by Moshman, Johnson 
and Larsen, the program priority is used as a 
weighting factor in scheduling and allocat- 
ing resources among competing alternative 
uses in the multi-program network. 

In general, the iterative use of multi- 
program level and program-level network 
methods provides a medium through which 
program and department level managers 
may devise integrated total plans. Optimized 
networks may be submitted by each program 
manager. In 1974, Woodworth and Dane 
found these networks could be merged into a 


multi-program network. Several multi- 
program network schedules may be devel- 
oped, given various assumptions about 
priorities and resources. These alternative 
multi-program schedules may then be exam- 
ined in staff meetings attended by each pro- 
gram manager and multi-program manager. 
The best multi-program schedule may then 
be selected, based on discussions and criti- 
cisms by everyone involved. Of course, sever- 
al iterations of the schedule may be required 
between the program and multi-program lev- 
el before an acceptable plan is developed. 

Influence on Program Performance 

Program completion may be strongly influ- 
enced by the company’s risk -failing propensi- 
ty, the customer’s decision process and the 
ability of the company to expand its organi- 
zation rapidly without losing its effective- 
ness. On some programs, these aspects may 
have more influence on performance. 

Network scheduling techniques, like PERT 
and CPM, are much alike in providing inter- 
dependencies, depth of detail, a critical path 
and slack. The swan technique provides sim- 
plicity and visibility through time scales that 
have been used for many years in bar charts. 

The choice between PERT and CPM depends 
primarily on the type of program and man- 
agerial objectives. The PERT is particularly 
appropriate if there is considerable uncer- 
tainty in program activity times, and it is im- 
portant to control the program schedule ef- 
fectively. On the other hand, CPM is particu- 
larly appropriate when activity times can be 
adjusted readily, and it is important to plan 
an appropriate tradeoff between program 
time and cost. 

Actually, differences between current ver- 
sions of PERT and CPM are not necessarily 
as pronounced as this section may convey. 
Most versions of PERT now allow only a sin- 
gle (most likely) estimate of each activity 
time. 
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When several small programs are to be 
scheduled, a multi-program network might 
be considered. In this situation, each pro- 
gram can be treated as a separate entity and 
the entire set of programs diagrammed and 
handled as one large network. The RAMPS 
computer program is convenient to apply in 
such a situation. The programs in the multi- 
program network should be importance- 
weighted or priority-constrained. This will 
determine which programs to schedule earli- 


er than others. Table 4 cites the strengths 
and weaknesses of network scheduling tech- 
niques. 

Line-of-Balance Technique 
Network scheduling techniques are used pri- 
marily in development and other one-time 
programs. The line-of-balance (LOB) tech- 
nique is used in repetitive activities like pro- 
duction. In production programs, LOB charts 
are particularly useful to balance inventory 


Table 4. Networking Technique: Strengths and Weaknesses 


CRITERIA 
1 . Accuracy 


STRENGTHS 


N/A 


WEAKNESSES 

The technique is as accurate as the 
activity-time estimates. The margin of 
error is generally less in construction than 
in development. 


Reliability 

N/A 

Compounded, unreliable estimates in a 
large program may lead to unreliable 
status information. 

Simplicity 

Brings simple order out of 
mass confusion. 

Concepts of slack and network families 
can be difficult to grasp. Computerized 
networking complicates the process; 
however, on a complex program without 
computerization for criteria 5 through 8, 
the strengths shown cannot be obtained 
readily. 

Universality 
of Program 
Coverage 

Very good for one-time 
programs like construction 
and development. 

Weak in production phase of life cycle. 
Not well-suited for quantity production. 

Decision 

Analysis 

Excellent for stimulating 
alternatives, especially when 
coupled with time-cost data. 

If computer based. 

Forecasting 

Excellent for forecasting 
ability to meet schedules. 

If computer based. 

Updating 

Easy to update estimates as 
progress information is 
received. 

If computer based. 

Flexibility 

Portions of network can be 
changed easily to reflect 
program changes. 

If computer based. 

Cost 


Because considerable data is required, it i: 


usually costly - especially if computerized. 
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acquisition with the production process and 
delivery requirements. 

A line-of-balance chart shows which control 
points need attention, not to maintain future 
delivery schedules. Using the LOB tech- 
nique, reporting to customers or top manage- 
ment is quick, inexpensive and graphic. 
Charts used for analysis and trouble shoot- 
ing are suitable for at-a-glance status report- 
ing. Without a computer controlled produc- 
tion process, the line-of-balance technique 
doesn’t lend itself readily to day-to-day up- 
dating. However, a monthly or weekly check 
usually keeps the process on schedule. 

A line-of-balance technique consists of four 
elements: (1) objectives of the program; (2) 
production plan; (3) current program status; 
and (4) a comparison between where the pro- 
gram is and where it’s supposed to be, strik- 
ing the line-of-balance (Figure 15). 

The first step in preparing the LOB is draw- 
ing the contract delivery schedule on the ob- 
jective chart (Figure 15-A), which shows cu- 
mulative units on the vertical scale and 
dates of delivery along the horizontal scale. 
The contract schedule line shows the cumu- 
lative units to be delivered over a period of 
time on the program. Actual deliveries to 
date (cumulative) are shown. The second step 
is charting the production plan (Figure 15- 
B). The assembly plan is a lead time chart. 
Select only the most meaningful events as 
control points in developing this chart. 

These main events can be given symbols that 
show whether they involve purchased items, 
subcontracted parts, or parts and assemblies 
produced in-house. Assemblies break down 
into subassemblies, which break down into 
parts or operations. Thus, one can develop a 
production plan for any part or level of as- 
sembly. 

The more steps that are monitored, the more 
sensitive and more complicated the chart be- 


comes. Generally, control points on a single 
chart should be limited to 50. If there are 
more than 50, subsidiary production plans 
can be used to feed the top plan. Thus, each 
chart can be kept simple and easy to under- 
stand. The shipping date of subsidiary charts 
is when a sub-program must be ready to join 
the overall schedule. 

On the production plan chart, each moni- 
tored step is numbered left to right. Step 1 
has the longest lead time. The shipping date 
is the highest numbered step. When two 
steps are done at the same time, they are 
numbered from top to bottom. 

The production plan chart shows interrela- 
tionships and sequence of major steps, and 
lead times required for each step. An under- 
standing of the manufacturing processes in- 
volved and sound judgment are required to 
know which step and how many steps must 
be monitored. 

The 12 control points in the production plan 
chart used as an example represent key tasks 
in manufacturing one lot of missiles. The 
plan indicates that fabricate ballistics shell 
(control point 1) must begin 24 work days be- 
fore government acceptance. Thus, this ac- 
tivity must begin 24 work days before Janu- 
ary 1 to meet the first scheduled delivery of 
five units by the end of December (see the ob- 
jective chart). The lead time for other control 
points can be related to the scheduled deliv- 
ery in a similar manner. 

In a five-day-week operation, a month is gen- 
erally recognized as having 22 work-days. 
Time for in-house transfer and storage must 
be allowed in addition to the processing time. 

To control production, the manager needs 
monthly status information for each control 
point. On the program status chart (Figure 
15-C), the bar for control point (12) shows 
that 14 units of the product have been accept- 
ed by the government. The bar for control 
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Figure 1 5. Line-of-Balance Charting 
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point (9) shows that 40 units of the guidance 
section have been assembled. The bar for 
control point (4) shows that in-house fabrica- 
tion has begun on 60 fins, but only 25 have 
been completed. The status at other control 
points are determined in a similar manner. 
Final deliveries (government acceptances) 
are shown month-by-month on the objective 
chart. 

To analyze how present status of each control 
point will affect future schedules, the line-of- 
balance (LOB) has been constructed. The 
line-of-balance represents the number of 
units of the product that should have passed 
through each control point to satisfy future 
delivery schedules. This line-of-balance is 
drawn above the bars on the program status 
chart to show the status of control points. 
Normally, the line steps down to the right. 

The difference between the line and the top 
of the bar for each control point is the num- 
ber of units behind or ahead of schedule. 
Thus, control point (12) is 16 units behind 
schedule, control point (9) is 5 units ahead of 
schedule, and control point (7) is 21 units be- 
hind schedule. Control point (12) is behind 
schedule now, May 1, because there is no 
lead time available for it. The main impact of 
control point (7) being behind schedule will 
be felt in 12 workdays, which is the lead time 
for control point (7). An insufficient number 
of assembled air vehicle bodies started into 
production on May 1. This will adversely af- 
fect final deliveries 12 workdays hence. All 
other control points can be analyzed in the 
same way. 

To recap, the line-of-balance is constructed in 
the following manner: 

• Select a control point, for example (7). 

• From the program (Figure 15-B) deter- 
mine the lead time, the time from the con- 
trol point to shipment point (12 work- 
days). 


• Using this number, determine the date 
the units now at the control point should 
be completed. (May 1 plus 12 workdays is 
May 16). 

• Find the point corresponding to this date 
(May 16) on the contract schedule line 
and determine how many units scheduled 
for completion this represents (41 units). 

• Draw a line on the program status chart 
(Figure 15-C) at that level (41 units) over 
the control point (7). 

• Repeat for each control point and connect 
the horizontal lines over the control 
points. The resulting line is the LOB, in- 
dicating the quantities of units that 
should have passed through each control 
point on the date of the study (May 1) if 
the delivery schedule had been met. 

Analysis 

Using the LOB charts in Figure 15, manage- 
ment can tell at glance how actual progress 
compares with planned progress. Analysis of 
charts can pinpoint problem areas. Delays at 
control point (7) in the example may have 
been causing final delivery problems 
throughout the contract. However, the pur- 
pose of line-of-balance analysis is not to show 
what caused the slippage in the shipping 
date, but to detect potential future problems. 

In the example, the government acceptance 
point is control point (12). The bar does not 
reach the line-of-balance; therefore, deliv- 
eries are behind schedule. Control points (10) 
and (11) are short. However, point (9) is on 
schedule. Since point (10) depends on points 
(8) and (9), we know control point (8) is the 
offender. Both points (7) and (8) are short, 
but there are more than enough purchased 
items at control point (6). What’s the problem 
with control point (7)? Trace it back to con- 
trol point (4), which is seriously short. It is 
obvious that not having enough completed- 
fins is holding up the whole progress. Control 
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points (2), (3) and (5) are short, but are not di- 
rectly responsible for the failure to meet the 
delivery schedule. The problem with the fins 
should be addressed before management at- 
tention is devoted to other short operations. 
The averages at control points (1) and (6) 
may be examined from the point of view of 
inventory control. 

Updating the charts requires a good status 
reporting system, which can be mechanized 
if the program is large and complex. A com- 
puter program, developed by the U.S. Army 
Management Engineering Training Activity 
(AMETA), Rock Island, Illinois, provides 
printouts of all information required on LOB 
charts. Actually, because the program pro- 
vides all information, printouts can be used 
by themselves. Charts are not required, but a 
graphic display of the information is usually 
desirable. 

The line-of-balance is a means for measuring 
actual progress against a scheduled objec- 
tive. It employs the exception principle. The 
four phases to line-of-balance are: the objec- 
tive, the program, program progress and 
comparison of program progress to the objec- 
tive. The statement of objective is presented 
in terms of the number of units/time period, 


number of units to be delivered, scheduled 
completion date or any other appropriate 
quantity /time combination (Figure 15- A). 

The graphical representation of the principal 
steps to be taken enroute to the objective — a 
modified Gantt Chart — is shown in the pro- 
duction plan chart (Figure 15-B). 

The graphical representation of the inven- 
tory of the stock status for the principal steps 
is shown in the program status chart (Figure 
15-C) with a vertical axis of the same units 
as those shown in the objective chart. 

Striking the line-of-balance involves trans- 
ferring points from the objective chart to the 
program status chart for the date being stud- 
ied. A program that is exactly in phase re- 
sults in a line-of-balance that intersects ev- 
ery bar of the program status chart at (or 
near) its top. 

Because the LOB technique is production ori- 
ented, it provides quick detection of bottle- 
necks in the production process. Manage- 
ment can then take appropriate action, such 
as increasing resources at each bottleneck. 
Table 5 summarizes strengths and weaknes- 
ses of the LOB technique. 
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Table 5. Line-of-Balance Technique: Strengths and Weaknesses (In Production) 



CRITERIA 

STRENGTHS 

WEAKNESSES 

1 . 

Accuracy 

Completion time estimates are 
good because work is repetitive; 
however, this may not be true 
early in the production phase 
of a program.* 

N/A 

2. 

Reliability 

Compares favorably with Gantt 
technique. 

N/A 

3. 

Simplicity 

N/A 

Construction of the line of balance is 
not always understood. 

4. 

Universality 
of Program 
Coverage 

N/A 

Well suited only for production phase 
of life cycle. Does not emphasize 
resource allocation directly. 

5. 

Decision 

Analysis 

N/A 

No capability to simulate alternatives. 

6. 

Forecasting 

Very good for indicating 
whether or not schedules 
can be met. 

N/A 

7. 

Updating 

N/A 

Considerable clerical effort is needed 
to update graphs. Computer 
processing can reduce this effort. 

8. 

Flexibility 

N/A 

Inflexible. When major program 
changes occur, all LOB phases must be 
redesigned. 

9. 

Cost 

Data gathering and computations 
can be handled routinely and at 
moderate expense. 



* Most of the production span time ( — 70 to -80 percent) consists of wait and move time. These 
times are usually less accurate than the standard times used for set up and run. Reporting ac- 
curacy is a key to the reliability of this technique. 


“Dost thou love life, then do not squander time for that’s the stuff life is made of. ” 

- Benjamin Franklin 
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Time Management 

Program managers are busy people, particu- 
larly those in the DoD and defense-related 
industry. Therefore, it is important that pro- 
gram managers manage time well. Some 
managers could be more productive, perhaps 
as much as 20 to 40 percent, by applying ef- 
fective approaches. 

This section concerns the aspects of time 
management related to programs: the pro- 
gram manager’s time reserve, a “now” sched- 
ule and value of time. 

In contractor performance measurement, 
much emphasis is placed on “management 
reserve,” the reserve budget controlled by 
the industry program manager. The program 
manager doesn’t know when or where this 
reserve will be needed. But the program 
manager in industry and his counterpart in 
DoD know it will be needed. 

A time reserve is needed as well as a budget 
reserve; the program manager needs a time 
reserve to accommodate unknowns that he 
will encounter. The use of time reserve 
should be approached with caution, especial- 
ly where it is visible, so as not to negate the 
value of the schedule plan and status for 
management use. 

Literature describing a program manager’s 
time reserve is scarce. Based on discussions 
with a sampling of managers of large and 
small programs, the main aspects of time re- 
serve are clear. 

• Most program managers establish a time 
reserve of about 10 percent. On a 40- 
month program, for example, a four- 
month time reserve would be established. 

• The time reserve must be held closely by 
the program manager, otherwise every 
manager on his program may think, “I 
know there’s a time reserve; I don’t really 
have to meet my schedule.” The program 


manager may place this reserve under 
“additional system tests” or another 
downstream activity. The point is, it 
shouldn’t be visible. (A built-in safety fac- 
tor between the manufacturing schedule 
and the delivery schedule is often used.) 

• A tough and disciplined approach to 
meeting the published schedule is re- 
quired from the start of a program in or- 
der to maintain the reserve and, conse- 
quently, to meet the program schedule in 
spite of slippages caused by the unknown 
unknowns (unk unks) that inevitably 
arise. 

A direct relationship exists between time 
and cost for any activity. This relationship 
takes into account the people, resources and 
method used, and considers the efficiency 
achieved. Generally, the least costly sched- 
ule is the current one. To speed up the sched- 
ule costs more; to stretch out the schedule 
also costs more. The sum of the direct and in- 
direct costs gives a U-shaped total program 
cost curve. The optimum schedule for imple- 
menting the program is the schedule corre- 
sponding to the minimum point on this 
curve. The relationship among direct, indi- 
rect and total program costs is shown in Fig- 
ure 16. 

Because schedule stability affects program 
costs, which may, in turn, affect technical 
performance, it is clear that schedule stabil- 
ity has a great deal to do with whether the 
program meets its cost and technical objec- 
tives. Unfortunately, budget constraints and 
other factors, like changes in quantities 
(items over which the program manager has 
no control) have often been imposed on a pro- 
gram with the comment, “Do the best you 
can.” 

When a schedule must be revised, the super- 
seded schedule is often discarded. If the new 
schedule is superseded, the process is repeat- 
ed. Often, the organization causing a slip in 
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Optimum 


Time 


Figure 16. Total Cost Analysis for Selecting "Optimum" Program Duration 


schedule becomes a repeat offender. The 
principal value of retaining a former sched- 
ule is in being able to identify the offender, 
thus making schedule slips less acceptable. 

The significance of maintaining a stable 
schedule is becoming more widely recog- 
nized. Appendix A describes the development 
of a master schedule and the importance of 
maintaining schedule discipline. 

While serving as under secretary of defense 
(research and engineering), Dr. William J. 
Perry said, “Our acquisition process is cau- 
tious, slow, and expensive. It now takes us 12 
years or more for development, production 
and deployment of a typical (defense) system, 
so that our lead in technology is lost by the 
time the system is deployed.” 


According to the late John H. Richardson, 
president of Hughes Aircraft Company, “A 
basic reason for adopting project (or pro- 
gram) management, when tackling the diffi- 
cult and unique tasks associated with devel- 
oping and producing a system, is to eliminate 
unnecessary delays in accomplishing the job 
at hand. Time is a resource in systems man- 
agement, to be treated with indifference or 
used well like any other resource. For pro- 
jects not yet in full swing, it is important to 
recognize that time has economic value, and 
that we may be taking time too much for 
granted.” Richardson cites historic reasons 
for stretching out program schedules. 

• Funding can create a problem. In hungry 
years the schedule is often stretched be- 
cause of reduced funding. 
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• A better product can be expected if it is 
more thoroughly debugged and tested. 
However, a system does not really get 
wrung-out until it is in the user’s hands, 
regardless of beforehand debugging. 

• Cost of concurrency (overlap of develop- 
ment and production), may lead to a deci- 
sion not to overlap program phases. Such 
a decision may be popular in many cases 
but it cannot be tolerated when the pen- 
dulum swings toward the importance of 
time; that is, when top management says 
“Get the system out the door; never mind 
what it costs.” 

Stretched-out schedules incur cost penalties 
because of inflation, additional engineering 
changes, and changes in program managers 
or other key program management officials. 
Another near-term cost results from the in- 
creased chance that a program will be can- 
celed because of obsolescence or competing 
technology. History shows that stretch-outs 
invite cancellation. Also, competing technol- 
ogy, which may get a toe-hold during a pro- 
gram stretch out, may lead to a program can- 
cellation. Long schedules with no opportuni- 
ties to incorporate improvements are a nega- 
tive factor when considering a new start. 

Delayed decisions increase costs. For exam- 
ple, waiting to acquire 90 percent of the facts 
bearing on a decision, rather than going 
ahead when 80 percent of the facts are in 
hand, is not usually cost effective. The sched- 
ule is prolonged when the decision is with- 
held. 

According to R. W. Peterson, former Du Pont 
executive, “All business men are concerned, 
and properly so, about the long time it takes 
to move a new development from its incep- 
tion to a profit status. But frequently forgot- 
ten is the fact that a month’s delay in the ear- 
ly stages of development is exactly as long as 
a month’s delay in the later stages. While it 
may seem innocuous to put off a decision for 


a month or two in the early years of the pro- 
ject (or program) with an uncertain future, 
that delay may turn out to be just as costly as 
is procrastination when the final decisions 
are made. In short, a sense of urgency is es- 
sential to decision making in all stages of a 
new venture, not just the latter stages.” 

A consideration having more impact on the 
value of a defense system, a point often over- 
looked, is the useful life of the system. Lead- 
ing producers of commercial and industrial 
products are aware of the importance of 
bringing a new product to market without 
delay to gain the greatest return on the costs 
of product development and production. 

Making use of time to increase the life of a 
system applies to defense systems as well as 
to commercial/industrial products/systems. 
Concentration on system or product cost, 
without considering the life of the resulting 
system or product, overlooks a key point: 
whether the buyer obtains value for each dol- 
lar. The most costly product, in terms of val- 
ue, is one appearing when it no longer fulfills 
a useful purpose, even though it has been 
produced at minimum cost. Each month ad- 
ded to the development and production of a 
new high-technology system or product tends 
to reduce by one month the operational life of 
the system or product. 

In spite of the 10 to 20 percent cost premium 
that may be paid for tight scheduling, as 
compared to orderly but stretched-out sched- 
uling, the longer resulting operational life 
usually provides greater economic value. 
This is looking at time only from the view- 
point of economics; i.e., acquisition cost per 
year of operational availability. 

Another way of looking at time is that de- 
fense system availability is survival insur- 
ance. An executive of a major shipbuilding 
company noted that, “the time we’re spend- 
ing waiting for our ships to come in ... is 
time we just may not have.” 
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Consideration of alternative plans and 
schedules will also help; e.g., if event so-and- 
so occurs, proceed with plan “A”; if event 
such-and-such occurs, proceed with plan “B”, 
and so on. Anticipation and preparation for 
most likely events, along with the tools de- 
scribed, and coupled with effective communi- 
cation of the plans, can change the manage- 
ment style from crisis management to skill- 
ful management. 

Planning and scheduling can do much to pre- 
vent running out of time and having to make 
the least desirable decision because of lack of 
time. Establishing a time reserve, a “now” 
schedule and recognizing the value of time in 
decision making all contribute to the man- 
ager’s repertoire of good tools. 

Sir Jeffrey Quill, manager of the British 
Spitfire Development Program, commented 
during a visit to the Defense Systems Man- 
agement College that, “After 1935, costs 
were not particularly important. What mat- 
tered was time. We worked three shifts a 
day. Everything was time. Quantity and 
time. It turned out that we probably pro- 
duced at the lowest cost too; but the emphasis 
was on time.” 

“I wasted time: now doth time waste me. ” 

- Richard II 

Recapitulation 

The program manager is responsible to top 
management for getting the job done on 
schedule and within the allowable cost. To- 
day, network based systems assist the man- 
ager in planning, scheduling and controlling 
the work to be accomplished— -often by people 
in separate organizations not under the man- 
ager’s direct control. The manager needs a 
plan that will provide a constant and up-to- 
date picture of the operation that is under- 
stood by all. 

Scheduling, cost and performance are major 
elements of concern to the program manager, 


who should be able to blend them to meet 
program objectives. When selecting a sched- 
uling method, the program manager can 
make a conscious tradeoff between the so- 
phisticated methods available and cost. 

Gantt charts can be used effectively for small 
programs and when program activities are 
not highly interconnected. Often, a Gantt 
chart is selected because, considering the 
benefits it will provide, network scheduling 
does not justify the additional cost. Figure 17 
illustrates the evolution of network -based 
systems. The differences between the CPM 
and PERT techniques result from the envi- 
ronments in which they evolved. 

The CPM arrow-diagram network evolved 
from activity-oriented bar charts. The arrow 
diagram resulted from linking the activities 
in a sequence of dependence, often without 
identification of the connecting points. Fac- 
tors leading to the CPM technique are a well- 
defined program, a dominating organization, 
few small uncertainties and a single geo- 
graphical location for the program. 

The PERT network evolved from a combina- 
tion of bar charts and milestone charts on 
which the milestones were identified as 
events, or specific points in time. The PERT 
network is heavily event-oriented. Factors 
that led to the development of the PERT 
technique are large programs with difficult- 
to-define objectives, multiple and overlap- 
ping responsibilities among organizations, 
large time and cash uncertainties, and wide 
geographic dispersal of activities and com- 
plex logistics. 

When network scheduling is justified, and 
you wish to choose one of the methods dis- 
cussed in this guide, be sure to consider that 
many network scheduling methods are com- 
puterized. Software packages are available 
commercially, or at DSMC, to cover many 
scheduling methods. 
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The principal points to be derived from this 

section include the following: 

• Schedule, time and cost are three major 
elements to control in any program. 
These can be in conflict and, tradeoffs 
may have to be made. 

• All programs involve planning, schedul- 
ing and controlling. During the planning 
phase, objectives, organization and re- 
sources are determined. During the 
scheduling phase — the phase with which 
this section is concerned — personnel re- 
quirements have to be determined; time 
to complete the work and the cost have to 
be estimated. During the control phase, 
the program’s progress in terms of time, 
cost, and performance have to be mea- 
sured. Necessary corrections have to be 
made to ensure achievement of the pro- 
gram objectives. 

• The activity oriented Gantt charts are 
useful when activities are not closely re- 
lated and the program is relatively small. 
The chart shows relationships among 
variables clearly and quickly, and focuses 
on situations needing attention. 

• The milestone charts, which are event- 
oriented and display start and completion 
dates, invite surprises because the pro- 
gram manager may not know the status 
until an event occurs or fails to occur. 

• The network displays how a program can 
be done and the schedule establishes how 
it is planned to be done. 

• A network identifies the critical path, 
slack (time an activity or event can be ex- 


tended and still be completed on time) 
and activities needing rescheduling. Ac- 
tivities on the critical path have zero 
slack and must be completed on time to 
prevent slippage of program completion 
date. 

• The PERT network-based scheduling 
method may use three time estimates for 
each event: most optimistic time, most 
pessimistic time and most likely time. 

• The CPM, a network-based scheduling 
method, uses a linear time-cost tradeoff; 
i.e., it adds the concept of cost to the 
PERT format. If necessary, each activity 
can be completed in less than normal time 
by crashing the activity for a given cost. 

• The line-of-balance (LOB) technique of 
scheduling is effective in manufacturing 
where a final assembly line is fed by 
many component lines, and delivery of 
end-units is required at predetermined 
specified intervals. The effectiveness of 
LOB is based on the design of the assem- 
bly plan. 

• Computer programs are available for 
network-based scheduling. Manual calcu- 
lations are feasible for small problems 
like those set forth in this section; howev- 
er, computer assistance is usually neces- 
sary for large, complex problems. 

• Network theory assumptions that activi- 
ties are independent, discrete and predict- 
able are not always appropriate in actual 
applications. The departure from reality, 
however, does not normally affect plan- 
ning and coordinating efforts on critical- 
path scheduling. 


Ill 




READINGS IN PROGRAM CONTROL 


Resources 

L. G. Fendley, ‘Toward the Development of a 
Complete Multi-Project Scheduling System,” 
Journal of Industrial Engineering, 1968. 

Ted Ingalls, “We’re Looking for a Few Good Peo- 
ple,” Program Manager, July-August 1984. 

K. R. MacCrimmon and C. A. Ryavec, An Analyt- 
ical Study of the PERT Assumptions, Memo RM- 
3408-PR, Rand Corporation, Santa Monica, CA, 
December 1962. 

J. H. Mize, A Heuristic Scheduling Model for 
Multi-Project Organizations, unpublished Ph. D. 
thesis, Purdue University, Lafayette, IN, 1964. 

J. Moshman, J. Johnson, and M, Larsen, 
“RAMPS - A Technique for Resource Allocation 


and Multi-Project Scheduling,” Proceedings of 
the 1963 Spring Joint Computer Conference, 
Spartan Books, Baltimore, MD, 1963. 

Russell W. Peterson, former Du Pont executive, 
Governor of Delaware, and White House advisor, 
“New Venture Management in a Large Com- 
pany,” Harvard Business Review, May-June 
1967, p. 72. 

John H. Richardson, Time Defeats Technology, 
Hughes Aircraft Company, Culver City, CA, date 
unknown. 

B. M. Woodworth and C. W. Dane, Multi-Project 
Network Analysis with Resource Leveling : State- 
of-the-Art and a Governmental Application, Or- 
egon State University, Corvalis, OR, 1974. 



an Analysis of cost overruns on defense acquisition contracts 


N95- 20749 


An Analysis of Cost Overruns on Defense 
Acquisition Contracts 

by David S. Christensen 


Donald J. Yockey, the former Under Secre- 
tary of Defense for Acquisition, has called for 
more realism in the defense acquisition pro- 
cess. More specifically, he has called for real- 
istic cost estimates. The hope is that more re- 
alistic estimates will help surface problems 
in enough time to resolve them. 

Based on a review of over 500 contracts, the 
Office of the Under Secretary of Defense for 
Acquisition has observed that once a contract 
is 15 percent complete it is highly unlikely to 
recover from a cost overrun. Despite this im- 
portant observation, contractor and govern- 
ment personnel often claim that their pro- 
grams are different. 

This article examines the history of cost 
overruns reported on 64 completed defense 
contracts. Its purpose is to formally test the 
observation of the Under Secretary. Results 
confirm the observation at the 95 percent 
level of confidence, and were generally insen- 
sitive to the contract type (price, cost), the 
contract phase (development, production), 
the type of weapon system (air, ground, sea), 
and the armed forces service (Air Force, 
Army, Navy) that managed the contract. 
After a review of terminology, concepts and 
related research for those unfamiliar with 
the area, the methodology, results and man- 
agerial implications are described. 

Background 

Jacques Gansler reports that the average 
cost overrun on defense acquisition contracts 
is 40 percent. Cost data on defense contracts 
are regularly reported on cost management 
reports prepared by defense contractors. 
These reports include the Cost Performance 
Report (CPR) and the Cost/Schedule Status 
Report. Department of Defense Instruction 
5000.2 requires the CPR on all contracts 
judged significant enough for Cost/Schedule 


Control Systems Criteria (C/SCSC). Signifi- 
cant contracts are research, evaluation, test 
and development contracts with estimated 
costs of $60 million or more, or procurement 
contracts with estimated costs of $250 mil- 
lion or more. Thus, a 40 percent cost overrun 
on a procurement contract that barely quali- 
fies as significant is at least $100 million dol- 
lars! 

The cost/schedule control systems criteria 
are not a system. Instead, they are minimal 
standards for contractors’ internal manage- 
ment control systems. The purpose of the cri- 
teria is to foster reliable decision-making by 
contractor and government personnel. One of 
the requirements is that data reported by the 
contractor be summarized from the same sys- 
tems that the contractors use for internal 
management. These and other requirements 
help ensure that the data submitted to the 
government is useful for decision making. 

Another requirement of the criteria is a dis- 
ciplined budgeting system. A time-phased 
budget of all the authorized work on the con- 
tract, termed the “Performance Measure- 
ment Baseline,” is developed by the contrac- 
tor. The baseline is simply the summation of 
budgets assigned to elements of work on the 
contract. Because each element of work has a 
schedule, the budget for the work is said to be 
“time-phased.” 

The time-phased budgets assigned to work 
elements, termed the “Budgeted Cost of 
Work Scheduled” (BCWS), form the basis for 
earned value measurement and reporting. 
Earned value, also termed the “Budget Cost 
of Work Performed” (BCWP), is the same 
number as BCWS. The only difference is 
when they are recorded. BCWS is recorded 
when work is planned to be accomplished, 
and BCWP is recorded when work is actually 
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accomplished. If work is accomplished at a 
time different than it is planned to be accom- 
plished, then a schedule variance is identi- 
fied. In a disciplined budgeting system, all 
significant variances are investigated in a 
timely manner. 

A schedule variance often signals a cost vari- 
ance. A cost variance is simply the difference 
between the budgeted cost of work performed 
(BCWP) and the actual cost of work per- 
formed (ACWP). As with the schedule vari- 
ance, the criteria require the timely investi- 
gation and reporting of significant cost vari- 
ances. The intent is that through the timely 
analysis of variances, problems will be cor- 
rected before they become serious. 

Figures 1 and 2 illustrate the relationship 
between the three basic data elements just 
described. The performance measurement 
baseline is the cumulative expression of 
BCWS. Against this baseline, performance 
(BCWP) and actual cost (ACWP) are mea- 
sured. Figure 1 illustrates the typical condi- 
tion of defense contracts: over budget and be- 
hind schedule. 


- - EAC • - • - 

1 



Figure 1. The Current Cost Overrun 
and Overrun at Completion 


A cost overrun is an adverse cost variance. 
Figure 1 illustrates two kinds of cost over- 
runs, termed the “current overrun” and the 
“overrun at completion.” The current over- 
run is the adverse cost variance to date. The 
overrun at completion is the difference be- 
tween the total budget for all the work on the 
contract, termed the “Budget At Completion” 
(BAC) and the estimated final cost of the con- 
tract, termed the “Estimate At Completion” 
(EAC). Note that the overrun at completion 
is an estimate until the contract is complet- 
ed. As shown in Figure 2, at the end of the 
contract, BCWP equals BCWS and the cur- 
rent overrun is the final overrun. 



Figure 2. The Final Cost Overrun 

The estimate at completion is an important 
number and is very controversial, largely be- 
cause there is literally an infinite number of 
possible EAC formulas. The criteria do not 
prescribe a particular formula or set of for- 
mulas; the choice is the contractor’s. The 
only requirement is that the estimate be ra- 
tional. 

Because rational people can disagree, the 
government will usually evaluate the rea- 
sonableness of the contractor’s estimate by 
computing a range of EACs. Unfortunately, 
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there is little guidance on what constitutes a 
reasonable range. As a result, the projected 
overrun at completion supported by the gov- 
ernment program office is usually higher 
than the contractor’s estimate. Because the 
government program office is necessarily an 
advocate of their program, their estimate 
may also be unrealistically optimistic. 

One way to assess the reasonableness of the 
estimated overrun at completion is to com- 
pare it to the overrun to date. If the overrun 
at completion is less than the overrun to 
date, then the contractor or program office is 
optimistically projecting a cost recovery. 
Such was the case in the A- 12 program. In 
April of 1990 the A-12 was in full-scale de- 
velopment and was 37 percent complete. The 
contractors’ reported overrun at completion 
was $354 million. The overrun to date was 
$459 million. Thus, the A-12 contractors 
were predicting a recovery of $105 million. 
Although this may seem optimistic, it is im- 
possible to know for sure because the A-12 
was canceled in January of 1991. 

Is such optimism justified? More specifically, 
is it unrealistically optimistic for the predict- 
ed overrun at completion to be less than the 
overrun to date? Based on a review of cost 
overrun data on completed contracts, the an- 
swer is that such optimism is unrealistic 
with 95 percent confidence. 

What Prior Research Says 
There has been some research into this issue. 
Wayne Abba and Gary L. Christie, senior 
analysts at the Office of the Under Secretary 
of Defense for Acquisition, have observed: 

Given a contract is more than 15 percent 
complete, the [final] overrun at completion 
will not be less than the overrun to date, 
and the [final] percent overrun at comple- 
tion will be greater than the percent over- 
run to date. 


This observation is based on a review of cost 
data on over 500 completed contracts. The 
analysts are quick to point out, however, that 
timely management attention to adverse cost 
variances can reverse them, especially early 
in the program. The problem has been a fail- 
ure to use performance measurement data 
proactively. 

The assertion of Abba and Christie is based 
on a casual review of over 500 completed con- 
tracts. The results of two empirical studies 
support the assertion. Both Kirk Payne and 
Scott Heise established that once a contract 
is 20 percent complete, the cumulative Cost 
Performance Index (CPI) does not change by 
more than 10 percent; in fact, in most cases it 
only worsens. (For example, in April 1990, 
the A-12 program was 37 percent complete 
and reported a CPI of 0.77. By September, 
the program was 47 percent complete and its 
CPI was 0.72.) 

As shown in Equation 1, the Cost Perfor- 
mance Index is a ratio of BCWP to ACWP. 

CPI = BCWP / ACWP 

A CPI that is less than 1 means that for ev- 
ery dollar spent, less than one dollar of work 
is accomplished. It follows that when the cu- 
mulative CPI is less than 1, the contract is 
experiencing a cost overrun, and because an 
unfavorable cumulative CPI only worsens, a 
contract is not likely to recover from a cost 
overrun. 

Therefore, if the predicted overrun at com- 
pletion is less than the overrun to date, the 
contractor’s estimated final cost of the con- 
tract (EAC) is unrealistically optimistic. 
This study further establishes these results 
by examining the cost overrun history on 64 
completed contracts extracted from the De- 
fense Acquisition Executive Summary 
(DAES) database. 
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Methodology 

The DAES database has received summary 
data on completed contracts since 1977. Pres- 
ently, data is summarized from Cost Perfor- 
mance Reports by government program of- 
fices and sent to the Office of the Under Sec- 
retary for Acquisition as quarterly DAES Re- 
ports. The database is a fairly detailed source 
of information on the cost performance of 
U.S. defense acquisition contracts. It is also 
reasonably accurate because most of the con- 
tracts in the database are C/SCSC compliant. 

For this study, a sample of 64 completed con- 
tracts was extracted from the database. Al- 
though the sample was purely judgmental, it 
is considered sufficiently rich to generalize to 
any C/SCSC-compliant defense contract. Ta- 
ble 1 shows cost overrun data by various 
categories considered relevant to this study. 

Based on the Under Secretary’s assertion 
and the results of prior research, four hypo- 
theses were tested (Table 2). For Hypothesis 
1, the average final cost overrun in dollars 


(FCO$) exceeds the average cost overrun to 
date (CO$). Hypothesis 2 is the same, except 
the overruns are expressed in percentages. If 
these hypotheses are correct with statistical 
significance, then recoveries from cost over- 
runs are improbable with a certain level of 
confidence. For this study, the hypotheses 
were tested at the 95 percent level of confi- 
dence. 

Based on the results of our prior research in- 
volving estimates at completion, it was ex- 
pected that the results of the testing may be 
sensitive to the contract completion point 
and other factors specific to the contracts in 
the sample. Therefore, the hypotheses were 
systematically tested at nine contract com- 
pletion points (10 to 90 percent at 10 percent 
increments) for various categories within the 
sample. The categories examined were the 
contract type (fixed price, cost), the contract 
phase (development, production), the generic 
type of weapon system, (air, ground, sea), 
and the armed forces service that managed 
the contract (Air Force, Army, Navy). 


Table 1 . Final Cost Overrun on 64 Completed Contracts 


Contract 

Category 

Number 

Overrun ($Millions) 

Overrun (Percent) 

Avg 

Min 

Max 

Avg 

Min 

Max 

All 

64 

36 

-3 

493 

18 

-3 

109 

Army 

28 

21 

-3 

46 

20 

-3 

46 

Air Force 

18 

49 

-2 

407 

19 

-1 

109 

Navy 

18 

47 

0 

493 

13 

0 

46 

Air 

43 

45 

-3 

492 

18 

-3 

109 

Ground 

13 

23 

7 

42 

21 

5 

45 

Sea 

8 

12 

0 

36 

12 

0 

38 

Development 

25 

38 

-2 

407 

21 

-1 

109 

Production 

39 

35 

-3 

493 

16 

-3 

46 

Cost 

23 

41 

-2 

493 

14 

-1 

46 

Price 

41 

34 

-3 

407 

20 

-3 

109 
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The remaining hypotheses are related to the 
results of the referenced CPI stability studies 
which established that the cumulative CPI 
tends to worsen from the 20 percent comple- 
tion point. Here the hypothesis was that the 
average cost overrun tended to increase. To 
test this hypothesis, the average cost overrun 
(CO) was regressed against percent complete 
(x): 

CO = a + fix 


Table 2. Hypotheses 


Hypothesis 

Interpretation 

HI: FCO$>CO$ 

Recoveries from 
cost overruns ($) 
are improbable 

H2: FCO% >CO% 

Recoveries from 
cost overruns (%) 
are improbable 

H3: (3$ >0 

Cost overruns ($) 
tend to increase 

H4: p%>0 

Cost overruns<%) 
tend to increase 


If the resulting slope coefficient ((1) is positive 
with statistical significance, then the hy- 
pothesis is accepted, which means that cost 
overruns tend to increase. In Hypothesis 3, 
the average cost overrun was in dollars; in 
Hypothesis 4, the average cost overrun was a 
percent. As with Hypotheses 1 and 2, Hypo- 
theses 3 and 4 were tested on the entire sam- 
ple and on various categories of the sample. 

Equations 3 and 4 define the current cost 
overrun and final cost overrun in dollars. 
Equations 5 and 6 define the overruns as per- 
centages. 

Current Overrun (CO$) = Cum ACWP - Cum BCWP 
Final Overrun (FO$) = Final ACWP - BAC 
Current Overrun Percent = 100* (CO$/Cum 0C WP) 
Final Overrun Percent = 100* (FO$/BAC) 


The cost overruns were averaged for each 
category of the sample by dividing the num- 
ber of contracts in that category into the total 
overrun for that category. The averaging was 
done at various stages of completion ranging 
from 10 to 100 percent complete, where per- 
cent complete was defined as follows: 

Percent Complete = 100* (Cum BCWP/BAC) 

Data earlier than the 10 percent completion 
point were not considered sufficiently reli- 
able. It can take as long as one year from con- 
tract award for the contractor to demonstrate 
C/SCSC compliance. Until then the data on 
the Cost Performance Report is suspect. 

Results 

As shown in the remaining tables, the hypo- 
theses were generally confirmed at the 95 
percent level of confidence. Table 3 shows the 
results of testing Hypotheses 1 and 2 on the 
entire sample of 64 contracts. Recoveries 
from cost overruns expressed in either dol- 
lars or as a percentage are improbable, espe- 
cially cost overruns experienced between the 
10 to 70 percent completion points. Between 
these points the difference between the final 
cost overrun and the overrun to date was sta- 
tistically significant at confidence levels well 
above 95 percent. After the 70 percent com- 
pletion, the current overrun percent is neces- 
sarily much closer to the final overrun per- 
cent because monthly expenditures typically 
decrease as the work nears completion. 

Hypotheses 1 and 2 were also generally con- 
firmed for the categories of the sample exam- 
ined. In short, recoveries from cost overruns 
on defense contracts are highly improbable, 
regardless of the contract’s type, the con- 
tract’s phase, the type of weapon system, or 
the armed forces service that managed the 
contract. 
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Table 4. Cost Overruns Tend to Increase 


Contract 

Cost Overrun ($ Millions) 

Cost Overrun (Percent) 

Categories 

Slope ((3) 

SE 

t 

Slope (B) 

SE 

t 

All 

0.325 

0.020 

16.13 

0.198 

0.009 

22.09 

Army 

0.186 

0.013 

14.27 

0.234 

0.016 

15.08 

Air Force 

0.407 

0.034 

12.11 

0.180 

0.005 

36.11 

Navy 

0.459 

0.021 

21.38 

0.159 

0.013 

12.20 

Air 

0.416 

0.022 

18.71 

0.210 

0.010 

20.30 

Ground 

0.168 

0.024 

7.06 

0.193 

0.018 

11.00 

Sea 

0.095 

0.008 

12.57 

0.139 

0.013 

10.37 

Development 

0.318 

0.024 

13.37 

0.232 

0.008 

29.12 

Production 

0.330 

0.019 

17.18 

0.176 

0.012 

15.08 

Cost 

0.393 

0.018 

21.40 

0.166 

0.015 

10.88 

Price 

0.287 

0.022 

12.93 

0.215 

0.006 

34.48 


SE = Standard error of slope coefficient with the intercept forced to zero; 
t = t statistic; ta = .05 df = 8 = 1.895 


Table 4 shows the results of testing Hypothe- 
ses 3 and 4, and confirms that cost overruns 
on defense contracts tend to increase. The 
slope coefficients were greater than zero with 
statistical significance for the entire sample, 
and for each category of the sample that was 
examined. 

Managerial Implications 
The results of this research show that recov- 
eries from cost overruns on defense contracts 
are highly improbable, and that cost over- 
runs tend to worsen as a defense contract 
proceeds to completion. This was found to be 
true regardless of the type or phase of the 
contract, the type of weapon system, or the 
armed forces service that managed the con- 
tract. The results are consistent with the re- 
sults of related research involving the stabil- 
ity of the Cost Performance Index, and con- 
firm the observations of senior analysts at 
the Office of the Under Secretary of Defense 
for Acquisition. 

These results have strong managerial impli- 
cations for the project manager; more realis- 


tic projections of the final costs are needed. 
When the projected overrun at completion is 
less than the overrun to date, the projected 
overrun at completion is too optimistic. For- 
mer Under Secretary of Defense for Acquisi- 
tion Donald J. Yockey commented in 1991 on 
this issue: 

We can't afford to understate, sit on, or 
cover up problems in any program — at any 
time — at any level. They must be brought 
forward. This includes not just “ show 
stoppers” but also “ show slowers.” I can’t 
stress this strongly enough. 

Without more realistic estimates, senior 
management may be lulled into a false sense 
of security about their programs and fail to 
take appropriate action to correct problems. 

Wayne Abba and Gary Christie, senior ana- 
lysts at the Office of the Under Secretary of 
Defense for Acquisition, have commented 
that although recoveries from cost overruns 
are improbable, they are possible, especially 
if management pays proper attention to 
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them. With proper attention adverse vari- 
ances have been reversed. 

Proper attention requires a timely and disci- 
plined analysis of variances as they are iden- 
tified. It also requires a proper culture. A 
“shoot the messenger” culture was partly re- 
sponsible for the delayed reporting of adverse 
information on the A-12 program. According- 
ly, senior management should make every 
effort to cultivate a healthy attitude regard- 
ing variance reporting. Managers are neces- 
sarily advocates of their projects. But this 
does not mean suppressing or delaying the 
communication of adverse information about 
their projects to senior decision-makers. 

It is not known if recoveries from cost over- 
runs on non-defense projects are also improb- 
able. Perhaps additional research can explore 
this issue. Technical and political problems 
that contribute to cost overruns on defense 
projects may not be relevant to non-defense 
projects; however, the “shoot the messenger” 
culture involved in the A-12 program is cer- 
tainly a potential problem in non-defense in- 
dustries. 

A related “cultural” factor that contributed 
to the cancellation of the A-12 was the natu- 
ral optimism of senior management. In testi- 


mony before Congress, Navy Secretary of De- 
fense Garret characterized the senior manag- 
ers involved in the A-12 program as “can do” 
people who did not admit to failure lightly. 
Although optimism has its place, it can be 
dangerous when it blinds the manager to the 
truth. 

Finally, social scientists like Barry Staw 
have extensively documented many real- 
world examples of “escalation error.” In 
these examples, the decision-maker is ex- 
tremely reluctant to cancel an ongoing pro- 
ject or switch to an alternative, despite exces- 
sive overruns or other compelling evidence 
that the project has failed or that the alterna- 
tive is superior to the present course of ac- 
tion. In some cases, the manager chooses to 
escalate commitment to the project by in- 
creasing the spending on the project. Re- 
searchers have attributed such behavior to 
psychological factors such as a myopic “can 
do” attitude or a need to “save face.” 

More recently, Chandra, Bushman and Dick- 
haut have suggested that escalation error is 
caused by the manager’s desire to protect 
his/her reputation in the managerial labor 
market. Given the adverse economic conse- 
quences of cost overruns, additional research 
in this area is needed. 
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Program Control on the Tropical Rainfall 
Measuring Mission 

by Dorothy J. Pennington and Walter Majerowicz 


The Tropical Rainfall Measuring Mission 
(TRMM), an integral part of NASA’s Mission 
to Planet Earth, is the first satellite dedi- 
cated to measuring tropical rainfall. TRMM 
will contribute to an understanding of the 
mechanisms through which tropical rainfall 
influences global circulation and climate. 
Goddard Space Flight Center’s (GSFC) 
Flight Projects Directorate is responsible for 
establishing a Project Office for the TRMM to 
manage, coordinate and integrate the var- 
ious organizations involved in the develop- 
ment and operation of this complex satellite. 
The TRMM observatory, the largest ever de- 
veloped and built inhouse at GSFC, includes 
state-of-the-art hardware. It will carry five 
scientific instruments designed to determine 
the rate of rainfall and the total rainfall oc- 
curring between the north and south lati- 


tudes of 35 degrees. As a secondary science 
objective, TRMM will also measure the 
Earth’s radiant energy budget and lightning. 

The complexities of managing an inhouse 
project are magnified by many non-GSFC in- 
terfaces, as shown in Table 1. The TRMM 
Project Office is responsible for managing 
the integration of all segments of this com- 
plex activity and providing a cohesive team 
that will deliver a fully functioning observa- 
tory within budget and schedule constraints. 
These interfaces require careful manage- 
ment and coordination of technical, schedule 
and budget elements. While the project office 
provides overall program planning, direction 
and control, the subsystem managers and in- 
strument suppliers implement project re- 
quirements at a detailed level. 


Table 1. TRMM Organization Responsibilites 
Component Responsible Organization 


Precipitation Radar (PR) 

TRMM Microwave Imager (TMI) 

Visible Infrared Scanner (VIRS) 

Clouds and Earth's Radiant Energy System 

(CERES) 

Lightning Imaging Sensor (US) 

TRMM Science Data and Information 

System (TSDIS) 

Mission Operations 

H-ll Launch Vehicle and Launch Services 

Science Team 


TRMM Project 

Engineering Directorate/numerous aerospace 
companies 

Japan/ NASDA/Toshiba 
TRMM Project/Hughes 

TRMM Project/Santa Barbara Research Center 
EOS/Langley Research Center/TRW 

TRMM Project/Marshall Space Flight Center 

Earth Sciences Directorate/General Sciences 
Corporation 

Mission Operations and Data System Directorate 
Japan/NASDA 

Earth Science Directorate, U.S. Universities, JPL, 
NOAA, Japan, Australia, Israel, France, Taiwan, 
Great Britain 


Project Management 
Observatory Subsystems 
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One immediate challenge to securing a suc- 
cessful TRMM mission is implementing pro- 
gram control systems that will ensure an Au- 
gust 1997 launch from Tanegashima Space 
Center, Japan. The August 1997 launch is 
critical; if TRMM is not launched on time, 
high levels of solar activity forecast for the 
late 1990s would result in a reduced mission 
life. This constraint, along with the limita- 
tion of biannual launch windows at the Tane- 
gashima Space Center, places top priority on 
schedule performance, but not at the expense 
of technical excellence, safety or cost. 

Program Control Overview 
The TRMM Program Contol staff has estab- 
lished a comprehensive Program Control 
System that includes schedule management, 
financial management, configuration man- 
agement and risk management. The Pro- 
gram Control System is not simply a comput- 
er program. Rather, it consists of a series of 
checks and balances in each of these areas 
that are designed to keep the entire manage- 
ment system integrated, as shown in Figure 
1. Four monthly reports reflecting analyses 
in the areas of schedule, finance, general 


business and risk management are generat- 
ed by the TRMM program control staff. 
These reports, called the Program Control 
Monthly Status Reports (PCMSR), are dis- 
tributed to TRMM technical and resources 
management and provide a current, com- 
plete analysis of all business issues and con- 
cerns. TRMM also conducts monthly status 
reviews with each of the subsystem, instru- 
ment and element managers. During these 
reviews, each manager is allocated approxi- 
mately 30 minutes to present technical, cost, 
schedule and manpower issues and concerns 
to the TRMM Project Manager. The impor- 
tance placed on communication, whether 
through these reviews or in the PCMSR, is 
one of the key reasons behind the success of 
the Program Control System. 

A major element of the Program Control Sys- 
tem is the logic network. Using the project 
work breakdown structure, the project plan- 
ners developed an end-to-end network that 
was baselined shortly after the TRMM Sys- 
tem Concept Review. This network, in con- 
junction with the mission specifications and 
agreements, provided the foundation for pro- 



Figure 1. Program Control Process 
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ject management to focus on the preparation 
of the budget estimates. Careful consider- 
ation was given to technical and schedule 
risks and tradeoffs while attempting to de- 
termine annual funding requirements. After 
the technical, schedule and cost baselines 
were established, the TRMM Configuration 
Control Board (CCB) was set up to systemati- 
cally consider all changes to the baselines. 
Finally, the risk management report was ini- 
tiated by the Program Control staff to pro- 
vide project management with an ongoing 
early warning system. Through this mecha- 
nism, actions to resolve cost, manpower, 
schedule and technical problems are quickly 
identified and implemented. Frequent com- 
munication between project management, 
subsystem managers, instrument suppliers 
and the program control staff is the key to 
maintaining these effective management 
systems. 

Schedule Management 
The scheduling function is centralized at the 
project level. The scheduling staff is assigned 
to the project office and coordinates with both 
GSFC and outside organizations responsible 
for the development of the TRMM spacecraft, 
instrument, and ground segments as well as 
overall system integration and test (I&T). 

The TRMM Program Control staff has devel- 
oped a comprehensive logic network for 
TRMM that integrates key work tasks and 
milestones from all elements within the 
TRMM system. For work being performed at 
GSFC, the schedulers prepare the subnet- 
works in coordination with the responsible 
subsystem and element technical managers. 
For work being performed outside of GSFC, 
schedule data is received from the contrac- 
tors’ scheduling systems and incorporated 
into the TRMM schedule data base. 

A sample portion of the logic network is con- 
tained in Figure 2. The information contain- 
ed in the activity boxes or “nodes” identifies 
the task description, activity duration in 


work days, and total slack (the amount of 
time an activity or event can be delayed be- 
fore it impacts launch readiness). With the 
use of TRMM's automated scheduling system 
for developing and maintaining the logic net- 
work, barcharts are easily generated. The 
bar chart corresponding to the logic network 
sample presented in Figure 2 is shown in 
Figure 3. These detailed schedules are “roll- 
ed up” to an intermediate level in order to 
summarize the schedule information for 
management. Figure 4 depicts how the 
Thruster detailed schedule is summarized 
within the Reaction Control Subsystem 
(RCS) Intermediate Schedule. This “roll-up” 
or schedule summarization capability, com- 
bined with the precedence relationships 
among the activities in the logic network, 
provide the framework to properly manage 
the vertical and horizontal schedule integra- 
tion and traceability on TRMM. 

For effective Program Control of TRMM, 
maintaining a schedule baseline is as impor- 
tant as maintaining a technical and cost 
baseline. Moreover, proper configuration 
management of the TRMM schedule is vital 
in order to accurately assess the impact of 
changes. TRMM’s formal schedule baseline 
is identified in the TRMM Project Schedule 
Baseline Document (PSBD). The PSBD con- 
sists of three parts: major project milestones, 
project control milestones and the Observa- 
tory integration and test schedule. The 
schedule for these milestones can only be 
changed with the approval of the TRMM 
Configuration Control Board. 

The major project milestones provide the 
framework for overall planning and schedul- 
ing for the TRMM spacecraft segment, in- 
strument segment, and ground segment de- 
velopments, system integration and test, 
shipping and delivery and launch site prep- 
arations. These milestones, depicted at the 
top of the Master Schedule (see Figure 5) con- 
sist of the System Concept Review (SCR), 
Preliminary Design Review (PDR), Critical 
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EARLY early TOTL 
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Figure 3. RCS Thruster Detailed Schedule Barchart 
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Figure 4. RCS Intermediate Schedule (1 of 2) 



Figure 4. RCS Intermediate Schedule (2 of 2) 
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Design Review (CDR), Pre-Environmental 
Test Review (PER), Pre-Shipment Review 
(PSR), and the Launch Readiness Review. 

The project control milestones are events 
which the TRMM Project Office considers 
critical. These include, but are not limited to, 
interface milestones such as the delivery of 


hardware or software between TRMM orga- 
nizational elements. Control milestones can 
also represent the completion of major stages 
of work within a given subsystem or element. 
More importantly, they are commitments by 
the responsible organizations to the TRMM 
Project Office to accomplish these events as 
planned. 
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Figure 5. TRMM Project Master Schedule 
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Next, the TRMM I&T schedule is included in 
the PSBD because it establishes the need 
dates for flight hardware and software. Con- 
siderable emphasis was placed on establish- 
ing the I&T schedule soon after the SCR in 
February 1991. Moreover, because all of the 
TRMM elements ultimately come together 
during integration and test, the I&T sched- 
ule has become the “hub” of the overall 
scheduling process. It is a key planning tool 
for all of the elements of the spacecraft, in- 
strument and ground segments. 

Since the logic network is a continuously 
evolving tool, it is not directly contained in 
the PSBD — only the project control miles- 
tones are. However, the logic network sup- 
ports the schedule baseline in that a target 
version of the network is maintained against 
which the current status is compared. This 
concept is illustrated in the sample barchart 
presented earlier (see Figure 4). The com- 
pressed black line below each activity bar or 
milestone represents the schedule baseline 
at the detailed work task level. This provides 
a correlation between the current schedule 
and the baseline. Unilateral changes to the 
logic network by the responsible subsystem 
or technical managers are permitted, pro- 
vided they do not impact the project control 
milestones or necessitate rephasing of the 
budget. 

Schedule status accounting on the TRMM 
Project occurs formally each month. Work al- 
ready underway or activities that should 
have started or been completed since the last 
accounting period are statused by determin- 
ing the percentage of work accomplished, the 
amount of time remaining to complete a 
task, or the new expected finish date of a 
task. For the work being performed at GSFC, 
the responsible subsystem technical manag- 
ers are interviewed by the schedulers in or- 
der to obtain schedule status. In this way, the 
schedulers receive not only the status, but 
also the rationale and issues affecting the 


status. Once the raw status is input into the 
logic network data base, it is processed, ana- 
lyzed and verified. This allows schedule is- 
sues to be identified, resolved or addressed 
before status is formally reported in the 
TRMM Monthly Project Review. For TRMM’s 
scientific instruments, schedule status is re- 
ceived from the instrumentors each month 
and analyzed prior to incorporation into the 
logic network. 

The key driver in the TRMM schedule is the 
August 16, 1997 launch readiness date. In 
addition to monitoring the actual progress of 
work toward launch readiness, the TRMM 
schedulers conduct a careful analysis of 
schedule slack. Total slack is a specific, 
quantitative and easily understood measure 
of schedule health. Figure 6 depicts the 
TRMM Total Slack Summary, which 
presents a high level overview of TRMM 
progress for a given month. The chart high- 
lights the key elements for the spacecraft, in- 
strument and ground segments. Each month 
the total slack for the worst case item within 
each TRMM element is elevated to the total 
slack chart. It is compared to total slack from 
the previous month, as well as the total slack 
for that item in the schedule baseline. The 
benefit of the total slack chart is that TRMM 
project managers can see the overall health 
of the TRMM project schedule at a glance. 

The schedule products such as bar charts and 
network diagrams are important Program 
Control tools for TRMM. When combined 
with a formal status process, they enable the 
TRMM Project Office to assess the progress 
of the TRMM schedule. As an early warning 
mechanism, the scheduling system provides 
a means to detect potential schedule prob- 
lems, implement workaround plans, or take 
corrective action in order to mitigate prob- 
lems. Scheduling products are tailored to 
various members of the TRMM team. Tools 
such as the Total Slack Chart and the Inter- 
mediate Schedules provide a way to summa- 
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Etomwit 
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Figure 6. TRMM Total Slack Summary 


rize a tremendous amount of detailed sched- 
ule data for TRMM project management. 
With this information, management can fo- 
cus on the key issues, critical paths and po- 
tential workarounds. At the working level, 
detailed schedule bar charts and logic net- 
work diagrams are excellent planning tools. 
In summary, the TRMM scheduling system 
provides reliable information to all levels of 
users. 

Financial Management 
A key feature of the Program Control System 
is cost and schedule integration. As with the 
scheduling staff, the financial staff is cen- 
tralized at the project level, although other 
GSFC organizations also provide financial 
support for TRMM subsystem managers. The 
main duty of the financial staff is budget for- 
mulation and execution. The logic network 
schedule serves as a basis for TRMM budget 
planning. Based on a detailed integration 


and test sequence, need dates for flight hard- 
ware and software have been precisely iden- 
tified. Budgets were formulated against the 
timeframe reflected in the schedules, as il- 
lustrated in Figure 7. 

By integrating cost and schedule planning, 
the project office has the capability to per- 
form what-if budget and schedule simula- 
tions. Civil servant manpower and travel 
budgets were also developed using the sched- 
ule as a guide in determining the correct 
phasing of requirements. In a dynamic bud- 
get environment, the TRMM Project is quick- 
ly able to isolate the impact of schedule de- 
lays, manpower shortages and travel cuts on 
the budget requirements. Similarly, when 
budgets are reduced, the integrated cost and 
schedule information provides a framework 
to quickly determine the scope of work that 
can be reprogrammed without having unde- 
sirable effects on launch readiness. 
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The TRMM Project has already used this sys- 
tem to identify numerous planned early- 
year, high-cost component purchases that 
could be deferred to later years, thereby alle- 
viating funding problems without jeopardiz- 
ing the integration and test schedule. 

Close coordination between the subsystem 
and element managers and the TRMM finan- 
cial staff ensures timely and accurate prep- 
aration of budget estimates and procurement 
requests. Since TRMM is an inhouse project, 
the procurement activities are not focused on 
several large prime contracts, as typically 
found in other GSFC projects. Instead, the fi- 
nancial and procurement staffs are responsi- 
ble for purchasing the components, parts and 
instruments that will come together as a 
complete observatory. These extensive pro- 
curement activities require detailed plan- 
ning and coordination to remain on schedule. 

The budget was developed for these procure- 
ments and supporting effort as discrete items 
at the Job Order Number (or work package) 


level. The budget requirements were then 
“rolled-up” through the project work break- 
down structure by month and fiscal year. 
This summarization ensures that budget 
data submitted to NASA Headquarters is 
based on the detailed estimates for the entire 
project. As part of the financial system, the 
TRMM financial staff has developed an ex- 
tensive contingency tracking system. Details 
of all changes in the budget baseline are 
maintained in the contingency (management 
reserve) tracking system as shown in the 
summary portion of Table 2. This provides a 
complete audit trail of all items funded from 
the contingency line item. 

In addition to budgeting and procurement re- 
sponsibilities, the financial staff analyzes 
contractor financial performance and en- 
sures that other members of the TRMM pro- 
ject team are kept abreast of financial issues 
and concerns. The TRMM Microwave Imager 
contract includes requirements for modified 
Performance Measurement System (PMS) 
reporting. 
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Figure 7. TRMM Cost/Schedule Integration 
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On a monthly basis, the financial staff pre- 
pares a quick look analysis of the PMS data 
in the TRMM Program Control Monthly Sta- 
tus Report. Analyses are also prepared for 
other contracts and for fiscal activity. 

Configuration Management 
TRMM’s integrated program control ap- 
proach also closely aligns cost and schedule 
management with configuration manage- 
ment (CM) activities. TRMM’s configuration 
management system provides a disciplined 
approach for controlling the changes to the 
requirements in hardware, software, perfor- 
mance, schedule and cost. Budget, schedule 
and technical requirements were established 
as integrated baselines early in the project's 
life. As changes to the established baselines 
occur, they are formally presented to the 
TRMMCCB. 

The CCB, composed of technical and admin- 
istrative representatives from each of the 
project disciplines, evaluates the positive or 
negative impact of each change on the bud- 


get, schedule, and technical baselines. With 
this integrated, accurate approach to cost 
and schedule assessment, the impact of engi- 
neering changes can be quickly and thor- 
oughly evaluated across the project. The 
TRMM Project Office has a goal to evaluate 
all changes within 45 days of the initial 
change request. A work progress indicator 
for the CM process has been incorporated 
into the Risk Management System. 

Risk Management 

Risk management is another key element of 
TRMM’s integrated program control process. 
The Risk Management System emphasizes 
detection and resolution of problems in areas 
identified as having risk potential. The sys- 
tem allows managers to identify program 
risks and to implement alternate plans to 
mitigate the impact of unresolved problems, 
as shown in Figure 8. Cost, schedule and 
technical risk parameters have been identi- 
fied for TRMM to quantitatively measure 
program health and ultimately program 
risk. 



Figure 8. TRMM Risk Management Process 
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Program control on the Tropical Rainfall measuring Mission 


Figure 9 shows the elements of the project 
that are tracked in the monthly Risk Assess- 
ment Report. Technical indicators include 
power, mass, data rate and mission life. Man- 
agement indicators include finance, sched- 
ule, configuration management, manpower 
and procurement. These risk indicators have 
been identified to provide a quantifiable goal 
against which progress is measured. 

Each indicator has three tolerance levels or 
alert zones used to indicate the level of risk. 
First, risk is classified as a major impact if 
the indicator’s performance reflects the exis- 
tence or imminent threat of major problems, 
concerns or similar severe impacts upon ac- 
complishment of project requirements. Sec- 
ond, the risk is identified as a potential im- 
pact if performance reflects the existence of 
problems, concerns or potential impacts on 
the project unless timely and effective action 
is taken. In the third category, the risk poses 


no negative impact on meeting TRMM cost, 
schedule and technical requirements. When 
an alert zone threshold is passed, an analysis 
is conducted by the responsible manager to 
determine the cause of the problem and a cor- 
rective action plan is generated to restore the 
indicator to the desired state. The Risk Re- 
duction Plan documents these products and 
provides an audit trail for the project to as- 
sign, track and close the corrective actions. 

Figure 10 illustrates the risk indicator sum- 
mary for the TRMM Configuration Change 
Requests. The project recognizes that failure 
to act upon change requests in a timely man- 
ner could affect the project’s ability to accom- 
plish cost and schedule goals. The alert zones 
reflect the project’s goals for the disposition 
of all change requests in 45 days. The accom- 
panying status shown in Figure 11 provides 
a monthly record of TRMM’s performance 
against these pre-established thresholds. 
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Figure 9. TRMM Risk Assessment Summary 
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Configuration Changes 

Purpose: To track the status of engineering 

changes (Class I) in terms of timely 
action to avoid schedule and/or cost 
impact. 

Data ground rules: 

• Track age of Configuration Change 
Requests (CCRs) 

• Change quantity measured by count of 
approved change logged into 
configuration control. 


Alert zones: 

□ No Impact 
or 

E3 Potential Impact 
or 

H Major Impact 


Age of CCR less than 45 
days 

Age of CCR between 45 
and 60 days 

Age of CCR over 60 days 


Figure 10. TRMM (CCRs) Indicator Summary 


When the assessment is unfavorable, a Risk 
Reduction Plan is generated (Figure 12) 
which analyzes the cause, impact and correc- 
tive action. The thresholds for the alert zones 
were set jointly by the responsible subsystem 
manager and the project manager and are in- 
tended to represent a reasonable goal for that 
indicator. These thresholds were sometimes 
adjusted several times in the preliminary 
months of the Risk Assessment Report until 
all parties felt that the appropriate goals 
were reflected. 

Figure 13 shows the risk indicator for the 
RCS schedule slack. This indicator, used for 
all subsystems and instruments, tracks slack 
trend status. Each month, the actual slack is 
compared to pre-established thresholds and 
risk reduction plans are generated as needed. 
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Log Numbor 

TRMM PROJECT RISK REDUCTION PLAN 

Problem Description Nsm# of Indicator 

Originator Date Phone Number 

Check the Alert Zone that applies: 

■Major Impact 3 Potential Impact □ No Problem, but has □ No Problem, but 

unfavorable trend RRP desirable 

Potential Impact: Cost Schedule Technical Performance 

Describe Problem 

1. Summarize problem, identify cause, quantify impact to cost, schedule technical performance. 

2. List hardware and/or software configured items affected. 

Corrective Action Plan (Be specific, include dates when problem is expected to be resolved, attach separate schedule if necessary.) 
Functional Manager Concurrence Project Manager Concurrence 

Figure 12. TRMM Project Risk Reduction Plan 



Figure 13, TRMM Reaction Control Subsystem Total Slack 
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Log Numbw 16 

TRMM PROJECT RISK REDUCTION PLAN 

Probtam Description RCS - Schedule Nw» of Indkalof ObMfVBlory-Scttadjta 

OrtgiMioc W«K Mabrosta Data VIS/93 Ptiooa Numb*/ 

dwelt Om Al*rt Zona that appttaa: 

■ Major Impact _2S_ B Potential Impact □ Mo Problem, but baa O No Problem, but 

unfavorable band «“* deelrable 

Potential Impact: Coal Schedule Technical Performance 

Describe Problem 

t. Summarize problem, identity cause, quantify Impact to coal, schedule technical performance. 

Delivery ol thruster delayed due to CCB 08 0066. total slack now * 1 7 which Is in ezeess ol Alert Zone t 

2. List hardware and/or software configured items affected. 


Corrective Action Plan (Be specific. Include dates when problem la expected to be resolved, attach separate schedule If necessary.) 
1. Release RFP by *73/93 need technical and schedule estimates from bidder's in order to determine II 20 5 month lead lime 
is teal Stic 

2 Begin investigating work-around plan lor 1ST assuming thruster Impact. 

Fur.tr, ooel Manager Concurrence Project Manager Concurrence 

Figure 14. TRMM Project Risk Reduction Plan 


In RCS, the January 1993 slack dropped to 
16 days due to a technical change in the 
thruster configuration. Since the first risk 
threshold of 32 days was passed, a Risk Re- 
duction Plan was generated (Figure 14). This 
problem was resolved in May 1993 by negoti- 
ating an earlier delivery with the vendor at 
contract award, with no additional cost. This 
action increased the thruster slack to 33 
days. With the thruster slack no longer in an 
alert zone, attention was then focused on the 
element with the least amount of slack, the 
Propellant Control Module (PCM). 

The risk management system has allowed 
the project staff to effectively use constrained 
resources to focus on problems which could 
negatively impact cost, schedule or technical 
objectives. Although the system requires a 
great deal of discipline, planning and team- 
work, the ultimate result focuses the entire 
project team on the critical problems. 


To date, the TRMM Project has succeeded in 
acheiving its cost and schedule goals and the 
TRMM Project Office can provide GSFC and 
NASA management with highly reliable sta- 
tus and forecast information. The TRMM 
Project Office’s proactive management ap- 
proach focuses on prevention rather than cor- 
rection. The ability to provide early warning 
and quick reaction analysis when changes 
occur has allowed the team to make informed 
decisions and to optimize positive results. 
TRMM technical, resource and management 
personnel clearly understand their role in 
aggressively managing their responsibil- 
ities. TRMM’s commitment to excellence, 
teamwork and communication will ensure 
the development of a high-quality satellite, 
delivered on schedule and within the ap- 
proved budget. This progressive manage- 
ment system is one of the TRMM Project’s 
contributions to improving NASA project 
management effectiveness and efficiency. 
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The spacecraft launched by NASA on 
November 3, 1973 to explore Venus and Mer- 
cury proved a notable success as a develop- 
ment project both in space and on the ground. 
This article on the development points out 
management approaches and techniques 
that kept schedules and controlled costs, the 
intent being to stimulate thought about how 
to do the same with future spacecraft and 
payloads. 

The Mariner Venus/Mercury ’73 (MVM ’73) 
project kept within its originally established 
goals for schedule, performance and cost. Un- 
derlying this development success was the 
availability of the Mariner technology. But 
meeting the goals demanded management 
determination, planning and discipline to 
make optimum use of state-of-the-art 
technology — on the part of people at NASA, 
JPL and The Boeing Co. (the contractor). 

Pre-project Highlights 

The earliest studies of the concept and scien- 
tific potential of a Venus/Mercury swing-by 
mission drew many to observe it could be the 
unique mission of the decade. It was the first 
to use a gravity-assist technique — taking ad- 
vantage of an unusual planetary configura- 
tion existing in 1973. Using the gravitation- 
al field of Venus, it was possible to swing an 
Atlas-Centaur-launched spacecraft onto a 
flight path to Mercury. Exploration of Mer- 
cury otherwise would not have been possible 
without using a much larger launch vehicle. 

The 1968 Planetary Exploration Summer 
Study conducted by the National Academy of 
Sciences (NAS) Space Science Board (SSB) 
endorsed this mission. The SSB suggested 
that the mission be planned around a single 
launch to make best use of the science funds 
available to NASA. 


Mission Objectives 

The following mission objectives, established 
by NASA following the Summer Study in 
1968, did not change during the program’s 
several years of design and development: 

• Primary. During the 1973 opportunity, 
to conduct exploratory investigations of 
the planet Mercury’s environment, atmo- 
sphere, surface, and body characteristics 
and to obtain environmental and atmo- 
spheric data from Venus during the fly- 
by. First priority goes to Mercury inves- 
tigations. 

• Secondary. To perform interplanetary 
experiments while the spacecraft flies 
from Earth to Mercury, and to obtain ex- 
perience with a gravity-assist mission. 

JPL had long experience with planetary pro- 
grams, but the opportunity for other Centers 
to participate in the program was not fore- 
closed. NASA’s Goddard Space Flight Center 
(GSFC) had plans for a Planetary Explorer 
spacecraft potentially able to do the mission 
and its approach was sufficiently attractive 
to invite further study. During the remain- 
der of 1968 and 1969, both GSFC and JPL 
studied their respective concepts; this early 
competition contributed to thoroughness of 
the early planning effort. 

The Scientists 

An innovative technique was used on MVM 
'73 to assure early involvement of the scienti- 
fic community with mission definition and 
preliminary design. In past missions, no ef- 
fective mechanism for the early detailed 
planning involvement of outside scientists 
had evolved, and selection of principal inves- 
tigators had been withheld until the comple- 
tion of mission-profile studies and early sys- 
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tem determinations. By the time the investi- 
gators were selected in those programs, 
many design features had already been es- 
tablished. 

For MVM ’73, selected scientists were invit- 
ed to participate in the early mission plan- 
ning. A group of scientists representing the 
several disciplines to be involved in the sci- 
ence payload was selected and formed into a 
Science Steering Group (SSG) in September 
1969. The scientists influenced the early mis- 
sion and spacecraft design, holding to a mini- 
mum conflict between mission constraints 
and science needs. 

Based on the positive results from these 
planning efforts, MVM ’73 was presented in 
the FY70 NASA budget as an Office of Space 
Science and Applications (OSSA) “new start” 
at a funding level of $3 million. An Authori- 
zation Conference Committee approved the 
project for inclusion in the FY70 authoriza- 
tion action, and funds were appropriated as 
requested. The scientific principal investiga- 
tors were then selected in a normal fashion 
after project authorization. 

Robert S. Kraemer, then head of planetary 
planning at NASA, pressed innovation in the 
early planning of MVM ’73. Kraemer later 
moved to the post of planetary program di- 
rector, with responsibility for implementing 
the project. 

The “Low Cost” Attitude 

The “low cost” attitude, so evident in the 
management of MVM ’73, developed early. 
The study teams were instructed to consider 
maximum use of established designs, residu- 
al hardware and existing capabilities. Very 
strict financial constraints were factored into 
payload planning. The SSG was requested to 
consider minimum cost experiments that 
would yield acceptable scientific data. The 
potential experiment proposers were advised 
to use existing designs for science instru- 
ments, to use flight-tested experiments 


wherever possible, and to consider modifica- 
tions only for high-payoff options. They were 
also to limit quality assurance, reliability 
and documentation requirements to that pre- 
viously applied to prior successful similar in- 
struments. GSFC and JPL established the 
mission and spacecraft baseline, developed 
preliminary implementation plans incorpo- 
rating the experiment approach being fol- 
lowed by the SSG, and made early cost esti- 
mates. JPL called on its extensive experience 
with Mariner spacecraft. Goddard proposed a 
spin-stabilized spacecraft of the Explorer 
class. 

JPL proposed to commit to a fixed cost to do 
the MVM ’73 mission in the system-contract 
mode. W.H. Pickering, JPL Director, advised 
OSSA in December 1969 that JPL could and 
would undertake the project for a cost not to 
exceed $98 million. 

The JPL Goal 

After a full briefing on the approaches by 
GSFC and JPL (proposed science return, 
spacecraft configurations, management 
modes, manpower and cost projections), 
OSSA chose JPL. In a letter to Dr. Pickering, 
assigning project management to JPL, John 
E. Naugle, Associate Administrator for 
Space Science, made this comment regarding 
mission cost: “A major concern has been and 
remains to be the total runout cost of the pro- 
ject. I am sure you are aware of the cost histo- 
ry for which estimates have ranged from ap- 
proximately $70 million to well over $100 
million. It is mandatory that the project be 
accomplished for a total cost not exceeding 
the $98 million quoted in your letter and 
strong efforts should be taken to reduce this 
figure.” This letter set the fundamental cost 
understanding between OSSA and JPL. 

The “Work Package” Concept 
JPL expertise in conducting flight projects 
predominantly involved obtaining spacecraft 
subsystems from industry thorough the JPL 
technical divisions with JPL accomplishing 
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the spacecraft systems functions. The major 
challenge faced by JPL in the MVM ’73 pro- 
ject was to utilize and adapt the fundamental 
JPL strengths to a system-contracting mode. 

A JPL team suggested a “work package” con- 
cept as the best means to transition from the 
use of subsystem contractors to a systems 
contractor. Appropriate elements of the JPL 
matrix organization prepared the work pack- 
ages. The project office exercised system 
technical direction, but the detailed defini- 
tion, monitoring and control of individual 
work units was performed by the appropriate 
JPL organizational element under the over- 
all coordination of the JPL project office. 

JPL also determined other factors important 
to implementing the project. It selected a cost 
contract with award fee. A specific JPL pro- 
curement group co-located with the project 
office would administer the system contract 
and other MVM ’73-related ones. It was de- 
cided that the JPL inhouse tasks should be 
given as much visibility and control as those 
of the system contractor. The constraint on 
resources dictated that all elements of the 
project, regardless of the performing organi- 
zation, be monitored in the same detail, and 
the risks balanced across all portions of the 
project’s activities. 

PAD, Procedures and Payoff 
The NASA project approval process entails a 
basic contract or understanding between the 
Administrator and the responsible Program 
Associate Administrator known as the Pro- 
gram Authorization Document (PAD). The 
initial PAD for the MVM ’73 project was 
signed on February 27, 1970. The objectives, 
technical plan, major support interfaces and 
procurement approach discussed in that PAD 
remained unchanged throughout the devel- 
opment. 

The JPL approach strongly exercised the 
Mariner heritage. MVM ’73 benefited not 
only from Mariner design derivation but also 


from residual hardware from past programs. 
The plan emphasized maximum use of exist- 
ing designs, hardware and software. This ap- 
proach saved perhaps 50 percent of design 
and development costs and perhaps 15 per- 
cent in hardware costs — a big payoff. 

The Cutting Edge 

The project team had lengthy discussions 
with JPL implementing organizations to 
identify the optimum way to meet cost con- 
straints. Control of cost-at-completion be- 
came a basic concept stressed by both the 
JPL and Headquarters offices in an attempt 
to avoid the less efficient, year-by-year fun- 
ding controls often followed in projects. The 
MVM ’73 project made it clear that each as- 
signed work unit was the total responsibility 
of the cognizant division and that responsi- 
bility for determining the least costly way to 
do the work rested squarely with the divi- 
sion. For each potential increase in cost, 
something had to be cut back. The JPL divi- 
sions almost invariably proposed specific 
cuts concurrent with notification to the pro- 
ject office of potential cost increases. 

Schedule Strategy 

The schedule adopted for MVM ’73 provided 
an unusually long period for advanced plan- 
ning and deferred this start of major con- 
tracts. This approach, unprecedented in 
launch-critical planetary programs, may 
have been the single most important factor 
in meeting cost goals. 

The added risk to the mission was offset by 
the increase in design time and better plan- 
ning of the fabrication effort. The effect was 
to establish a “most cost-effective” approach. 
The greatest number of people worked on the 
project for the shortest period of time. (Axi- 
om: the shorter the schedule, the less the 
cost.) 

Once adopted as a project philosophy, delay 
in implementation was applied to all aspects 
of the project. The systems contract was de- 
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layed three months beyond the schedule con- 
sidered minimal by many. Other subcontract 
work was released on a schedule that limited 
the work time to a prudent minimum. A “sin- 
gle thread” approach was followed in the 
spacecraft design where options were stud- 
ied, one was adopted, and the work started 
without carrying parallel efforts. Mission op- 
erations work was held off beyond the sched- 
ule previously considered to be optimum. 
Flight operations crew training was held off 
as long as possible. And it worked! There 
were no major schedule slippages, no serious- 
ly late deliveries of equipment and no ex- 
traordinary workarounds. 

“Do Only the Essential” 

The philosophy of “Do Only the Essential” 
became a discipline among project partici- 
pants. To challenge the need for each opera- 
tion, each added procedure, each piece of spe- 
cial equipment, and each separate design, re- 
dundant feature or test became routine. If a 
function, part, or operation was determined 
to be needed, then the search went on to see if 
hardware was available from other projects, 
or if the process had been developed by some- 
one else. If the part or process was not avail- 
able, then there was an attempt to use avail- 
able designs. 

This discipline was not only applied by the 
JPL managers but by Boeing as well. The 
Boeing spacecraft program manager proved 
extremely resourceful in identifying short 
cuts, reductions in paperwork, and unneces- 
sary redundancy — the cost-type contract not 
withstanding. The list of hardware and effort 
saved through this effort is too lengthy to dis- 
cuss here, but the savings extended to every 
area of the project effort. 

One unusual saving is notable. The project 
team encouraged a local college, assisted by 
several other colleges and high schools, to 
produce the spacecraft models, which often 
cost more than $100,000. The project gained 
all the models required, the students and 


schools gained good experience from their 
work on an interesting task and NASA saved 
dollars and encouraged local community in- 
terest and support. 

Project Team 

The most important ingredients to project 
success were the attitudes and skills of the 
people assigned to manage it. JPL’s exper- 
ience in dealing with a system contractor 
was limited to Surveyor, and by 1970 rela- 
tively few JPL people had been involved in 
the early stages of that project. The person 
most familiar with its operations was Walker 
E. “Gene” Giberson, who had been Survey- 
or’s project manager. He was appointed 
MVM ’73 project manager in January 1970. 

Giberson assembled a small team of indi- 
viduals, each selected on the basis of his past 
project experience and his willingness to 
work within firm budget allocations. The key 
members of this team included V. C. Clarke, 
Jr., mission analysis and engineering man- 
ager; J. A. Dunne, project scientist; J. R. 
Casani, spacecraft system manager; J.N. 
Wilson, assistant spacecraft system manager 
and N. Sirri, mission operation system man- 
ager. This team, trim in size yet representing 
broad experience, represented the core of 
MVM ’73 project management. 

The Guidelines 

At first, the team spent considerable time de- 
veloping the project’s operating concepts and 
indoctrinating everyone involved with the 
organizational and project philosophy. They 
set and held to the following guidelines 
throughout the project: 

• Establish early project guidelines, objec- 
tives and constraints 

• Use a small staff for planning 

• Prepare detailed plans and tasks before 
initiating a contract: 

- Specific and detailed RFPs 
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- A careful tradeoff assessment between 
JPL and contractor furnished equip- 
ment 

- Use of existing documents, reports and 
systems 

- Careful selection of fee approach 

• Establish cost-at-completion planning, 
budgeting and emphasis 

• Secure all contracts before starting work 

• Keep work and budget plans up-to-date 

• Exercise organizational impedance 
matching and communications 

• Maximize technical interaction 

• Use the concept of cognizant work unit 
engineer 

• Hold frequent face-to-face meetings of op- 
erating managers 

• Identify and resolve problems promptly 

• Make periodic status and performance re- 
views 

• Indoctrinate all involved with cost goals 

- Instill cost consciousness 

- Make cost goals believable 

- Develop a clear understanding of the 
cost-control system 

• Bring manpower onto the project and 
move it off in a timely manner 

The Hot Seat 

The Headquarters Program Office/Center 
Project Office interface can be extremely 
critical to the success of a project. If the pro- 
gram manager and project manager have dif- 
fering ambitions and objectives or, as oc- 
curred in some instances, an adversarial re- 
lationship, the project can suffer. N. William 
Cunningham, the Headquarters program 


manager, and Gene Giberson, the JPL pro- 
ject manager, enjoyed an open and forthright 
relationship, a cornerstone of a sound man- 
agement structure. 

The person on the “hot seat” for cost manage- 
ment is, however, the project manager. The 
project manager is the one most responsible 
for establishing the attitude and the frame- 
work for the daily tradeoffs of cost, perfor- 
mance and schedule where it is most essen- 
tial to maintain a proper perspective. With- 
out his cost consciousness, his basic approach 
to costs, MVM 73 would not have enjoyed its 
obvious cost success. This cost attitude is the 
more unusual since NASA had previously 
stressed technical performance and schedule 
requirements over cost as a discipline. 

The Science Steering Group selected in Sept- 
ember 1969 held its final meeting in March 
1970. In its report, the SSG recommended a 
minimum science payload composed of a 
plasma science experiment, a magnetometer, 
an infrared radiometer, an ultraviolet spec- 
trometer, a television system and an energet- 
ic particles experiment. 

One of the tasks of the SSG was to make a de- 
tailed cost estimate for each potential 
experiment — including design, development 
and fabrication costs of the hardware, cost of 
personnel support for launch and mission op- 
erations, and cost of data analysis and inter- 
pretation and publication of results. These 
cost estimates, plus a project estimate for in- 
tegrating the instruments into the space- 
craft, shaped the first science budget for the 
project at $13 million. 

An Announcement of Flight Opportunity 
(AFO) issued in March 1970 invited propos- 
als for experiments. It stressed the intent to 
select only proven flight-qualified instru- 
ments. The AFO also stressed the desire to 
minimize documentation and stated the in- 
tent of JPL to monitor development of the in- 
struments only at the interface level. 
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Forty-six proposals were received and evalu- 
ated. After ranking them in terms of science 
excellence, technical and engineering re- 
quirements, cost and system integration, the 
program office recommended seven payloads 
to the OSSA Associate Administrator. The 
payload cost estimates went as follows (in 
millions of dollars): 


Television 

$6.226M 

Radio science 

0.500 

Ultraviolet 

0.575 

Infrared 

0.928 

Magnetometer 

0.688 

Energetic particles 

0.383 

Plasma science 

0.945 

Total 

$10,245 

Instrument integration 

2.355 

Total 

$12,600 


To each of the principal investigators select- 
ed, Dr. Naugle addressed this comment: “I 
must emphasize, once again, that the total 
negotiated figure (dollar cost as selected) 
cannot be exceeded. Accordingly, I have in- 
structed the JPL project office that in the 
event of an anticipated cost overrun, their al- 
ternatives will consist of helping you to re- 
duce the scope of your experiment, or recom- 
mending its termination.” 

Science and Dollars 

Whereas most past selections had been con- 
sidered final at the time of announcement, 
the letter from Dr. Naugle clearly showed 
that the selection was to be considered tenta- 
tive until the investigators and JPL complet- 
ed negotiations. A process of fact-finding and 
negotiation between JPL and each of the sci- 
entific investigators followed, which resulted 
in well-defined relationships before the ma- 
jor development effort commenced. 

It was made clear in the selection and negoti- 
ation process that the principal investigator 
was responsible for the implementation and 
development of the investigation, including 
the instrument. The project office followed 


through on the intent to control principally 
at the instrument/spacecraft interface level. 
The systems contractor was responsible for 
integration of the instruments into the 
spacecraft. One innovative technique re- 
quired the systems contractor to “sign off’ on 
changes to experiment interface drawings, 
although the contracts for the experiments 
were between JPL and the investigator. This 
technique provided greater assurance that 
the systems contractor was aware of the 
latest configuration of the experiment hard- 
ware, and helped avoid surprises at the time 
of integration. 

Dr. Naugle views MVM ’73 as the most suc- 
cessful development of scientific instruments 
within tight cost constraints. The addition of 
the experiment integration costs to delivered 
cost brings the total for science very close to 
but within the original budget of $13 million. 

Meeting payload cost goals begs the question 
whether controls compromised the science 
investigations. A detailed review of the de- 
velopment history of each instrument clearly 
demonstrated that not only was there no 
compromise of the investigations during de- 
velopment, but that significant capability 
was added to several investigations. Any sci- 
ence compromise on MVM ’73 reflects direct- 
ly the original constraints established before 
experiments were selected. The decisions to 
tightly constrain payload costs, to fly only 
proven instruments and to apply go/no-go 
cost restrictions on instrument development 
are serious policy decisions to be carefully 
weighed. They cannot be applied to every 
payload but they paid off in MVM ’73. 

NASA and JPL held an industry briefing in 
February 1970 to apprise companies of the 
goals and constraints of the MVM ’73, to pro- 
vide detailed technical and program informa- 
tion for early planning, to encourage compe- 
tition, and to enlist industry’s help in deter- 
mining an optimum role for a system con- 
tractor; 41 firms attended the briefing. 
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JPL asked the companies for suggestions re- 
garding implementation of the systems con- 
tract approach; separate day-long meetings 
were held with the most interested competi- 
tors to discuss their suggestions. During 
these meetings, the companies made recom- 
mendations on contract scope, roles and rela- 
tionships, Mariner technology transfer, con- 
tract type, GFP handling and other areas 
they believed were important to the effort. 

A procurement plan evolved in which the 
systems contractor would have the major role 
(1) to design, fabricate, assemble, and test 
one flight spacecraft, one test spacecraft, as- 
sociated test models, test and support equip- 
ment and appropriate spares; and (2) to pro- 
vide level-of-effort support to JPL in mission 
analysis and engineering, JPL subsystems 
activities and mission operations. 

RFP Features 

The JPL project definition effort had been 
proceeding for a year at the time the Request 
for Proposals (RFP) was issued. The result of 
that effort was a very detailed RFP. It was an 
extensive compendium explaining project ob- 
jectives, organization and implementation; 
schedule, control dates and documents; work 
breakdown structure; spacecraft design sum- 
mary; scope of contract and general descrip- 
tion of work; JPL / contractor relationships 
and mission operations. Its most unusual fea- 
tures included: 

• A spacecraft systems specification which 
attempted to state only minimum re- 
quirements. 


• The predetermined intent to divide all 
work into discreet work units (which al- 
lowed separation of responsibilities and 
facilitated work description, understand- 
ing, negotiation and JPL monitoring). 
The definition of each work unit was writ- 
ten in a standard format. 

• The request for firms to propose overhead 
cost ceilings, 

• The request for baseline and alternate 
cost proposals to get the best cost mix be- 
tween JPL and contractor-furnished 
equipment. 

• A call for incentive proposals which gave 
heavy emphasis to cost, but also stated 
strong preference to award fee. 

• Emphasis on minimum documentation 
and maximum use of procedures, forms, 
techniques, etc. the contractor currently 
used. 

• Detailed documentation covering Mari- 
ner ’69 hardware, Mariner ’71 hardware, 
and other JPL-furnished equipment, 
along with drawings, schematics, pro- 
cesses and procedures to assure full use of 
the Mariner heritage and facilitate cost 
estimates. 

Four proposals were received. The Source 
Evaluation Board presentation was made to 
the NASA Administrator on April 28, 1971, 
and The Boeing Co. was selected as the sys- 
tems contractor. 


CY 1971 CY1972 

Category of Negotiated Per Negotiated Per 


Indirect Exoense 

Contract Actual 

Contract Actual 

Engineering 

$3.94 

$3.74 

$4.14 

$3.88M 

Manufacturing 

4.99 

5.08 

5.24 

4.97 

Productive Material 

10.5% 

7.9% 

10.5% 

6.7% 

Subcontract Material 

6.1% 

5.5% 

6.1% 

3.6% 

Area Administration 

15.1% 

14.35% 

15.1% 

11.9% 

Group Administration (remote) 

9.6% 

9.75% 

9.6% 

7.8% 
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Holding Out for a Firm Negotiated 
Contract 

The pressure to award the contract and com- 
mence work was very strong following the 
April selection, but the project manager and 
contract manager held out for a firm negoti- 
ated contract before allowing work to be 
started. Within six and one half weeks after 
selection, negotiations were completed and a 
definitive contract was awarded. Work start- 
ed on June 17, 1971. 

The cost-plus-award-fee contract emphasized 
the contractor’s complete responsibility to 
meet the spacecraft system performance re- 
quirements. The effort was divided into work 
units, each assigned to a manager within 
The Boeing Co. The work units were com- 
patible with both JPL’s technical division or- 
ganization and Boeing’s project structure. 

Controlling Overhead 

A serious concern in systems contracting had 
been the inability to predict overhead costs. 
The parties agreed that a ceiling on overhead 
costs would be negotiated into the contract. 
Such ceilings on overhead are unusual in 
normal circumstances, and all the more so in 
this case, considering the depressed economic 
situation The Boeing Co. faced in the spring 
of 1971. The ceiling on overhead never was 
invoked because Boeing actually underran 
the negotiated overhead cost. 

Strong cost incentives were negotiated into 
the contract and a process for evaluation and 
award emphasized performance and cost con- 
trol. The award fee provisions and the system 
employed to carry them out appear to have 
been effective in contributing to the contrac- 
tor’s performance. Benefits included these: 

• Boeing’s spacecraft program manager 
had the opportunity to increase the fee 
significantly. The award fee structure al- 
lowed broad latitude in the approach to 
cost and performance tradeoffs. 


• The process enforced periodic, results- 
oriented evaluations and communications 
at all levels. The process and the resul- 
tant dialogue tended to remove the obsta- 
cles that stand in the way of the natural 
motivation to do a good job. By clarifying 
goals, establishing emphasis, eliminating 
misunderstandings and highlighting 
problem areas for mutual attention, ob- 
stacles were removed or reduced. 

• Attention of the contractor’s top manage- 
ment was obtained by the formal feed- 
back process (briefings supported by let- 
ters). 

• The discipline of the award fee evaluation 
process improved JPL’s internal commu- 
nications at all levels, including top man- 
agement on the award fee review board. 

Tight Control 

JPL has a reputation in the industry for ag- 
gressive contract management, often ex- 
pressed as complaints of “too tight control” 
by subcontractors. But the JPL system 
proves effective in assuring performance. 

In MVM '73, change orders were kept to a 
minimum throughout the contract and were 
negotiated into the contract promptly after 
issuance. Project office personnel monitored 
Boeing’s work very closely. The work unit 
breakdown made it possible for cognizant 
JPL engineers to thoroughly understand the 
job, follow its progress in detail and identify 
potential problems early. 

Early identification of problems coupled with 
open, candid discussions among The Boeing 
Co. and JPL managers were basic contribu- 
tors to the success of the project. D.T. Gant, 
contracts manager, L.V. Burden, financial 
manager, and L.M. Bates, cost analyst, who 
were collocated in the project office, effective- 
ly kept the project managers alert to unex- 
pected deviations. 
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The NASA Management Audit Office, not 
noted for its approbative descriptions of 
NASA operations, gave this appraisal: “In 
our opinion, the JPL surveillance of the con- 
tract, its assignment of capable and motivat- 
ed personnel to monitor the performance of 
MVM ’73 on a full-time basis, and the appar- 
ent stringent cost controls implemented by 
The Boeing Co. before contract award, and 
retained throughout the program, contribut- 
ed to Boeing’s successful cost performance 
under MVM ’73.” 

Good Communications 
Stressed by the managers, good communica- 
tions led to early anticipation and resolution 
of issues and the timely availability of data 
for decision-making. Some of the techniques 
used to assure communications included: 

• A weekly Agreement /Disagreement Log, 
maintained by work unit personnel and 
reviewed by the JPL spacecraft system 
manager and The Boeing Co. spacecraft 
program manager. 

• Weekly face-to-face meetings between the 
systems contractor, systems manager and 
the systems contractor program manager. 

• A weekly summary of agreements and 
formal tracking of action items. 

• Daily meetings between The Boeing Co. 
test and operations representatives and 
the JPL resident staff during the system 
test period. 

• Weekly “Problem TWX.” 

• Formal monthly progress reviews to give 
an overview and detailed status and plans 
with particular emphasis on problems. 

• Easy access to The Boeing Co. and JPL 
top management (above the level of pro- 
ject personnel). 


• Attendance at award fee briefings by Boe- 
ing’s top management. 

• An extensive and definitive award fee let- 
ter and briefing, held not later than 15 
days after the end of each period. 

• Rapid escalation of significant problems 
to the appropriate management level for 
resolution. 

None of these actions should surprise good 
managers, but taken together, they may not 
be commonplace. These combined techniques 
greatly helped the MVM ’73 project meet its 
goals. 

Highlights of Contractor Performance 
The Boeing Co. faced an uncertain general 
business position at the time the MVM ’73 
project contract was issued. Major reductions 
had been made in Boeing’s commercial air- 
plane operations, and significant reductions 
in employment had been made at Boeing 
Aerospace Co. 

Despite the drastic reduction in backlog and 
direct workload, Boeing was able to reduce 
overhead costs and even underrun the over- 
head projections on the MVM ’73. The aero- 
space industry and its government customers 
are conditioned to the increase of overhead 
runs when the direct base decreases. This 
“fact” is considered by many to be axiomatic 
and inviolate; overhead costs are regarded as 
“fixed” or unalterable and necessary to sup- 
port the base for doing business. The exam- 
ple of Boeing’s experience in 1970 and 1971 
could be a good case study in ways to reduce 
overhead expense as the direct base de- 
creases. 

E. G. Czarnecki served as The Boeing Co. 
MVM spacecraft program manager from the 
early proposal phases in 1970 through early 
1973. H. Kennet served as deputy program 
manager and succeeded Czarnecki. Their 


145 



READINGS IN PROGRAM CONTROL 


participation contributed immensely to the 
success of MVM ’73. They have reviewed 
their experience, and underscored these 
management concepts and techniques em- 
ployed on MVM ’73: 

• Spacecraft requirements must be defined 
clearly and early. 

• Match people (skills) to work unit tasks. 

• Use the “cognizant work unit engineer” 
concept. 

• Select the baseline configuration early. 

• Implement a system of program reviews 
and reporting with joint chairmanship by 
contractor and customer. 

• Define and assess technical performance, 
schedule and cost risks and develop work- 
around plans. 

• Educate key personnel in the company’s 
cost accounting system so that when 
tradeoffs and decisions are to be made, all 
factors are properly considered and their 
true impact on cost understood. 

• Shorten and improve communications 
through collocation and program organi- 
zation. 

• Establish organizational relationships 
(e.g., JPL/Boeing) and communication 
channels early. 

• Motivate people through performance as- 
sessment, promotion, compensation and 
achievement awards. 

• Emphasize cost trades during design. 


• Ensure that only essential work is accom- 
plished. 

• Use an objective performance measure- 
ment system. 

• Rely on each cognizant work unit engi- 
neer for early identification, reporting 
and, when feasible, problem resolution. 

• Use dedicated manufacturing and test fa- 
cilities. 

• On-load and off-load manpower in a time- 
ly fashion. 

• Use recovery (“tiger”) teams to work 
problems. Teams of specialists from out- 
side the program can be assigned prob- 
lems and provide instant expertise with- 
out a continued expense to the program. 

A Postscript 

The MVM ’73 spacecraft (Mariner 10) was 
launched on November 3, 1973. A number of 
problems developed early in the flight, but 
none degraded the mission and none was the 
obvious result of actions taken to control 
cost. The spacecraft reached Venus on Febru- 
ary 5, 1974, and returned a full set of scienti- 
fic data, including more than 4,000 pictures. 
The gravitational attraction of Venus altered 
the spacecraft’s flight path as planned, 
swinging it toward Mercury. The spacecraft 
passed within 500 miles of Mercury’s surface 
on March 29, 1974, and returned the first 
close scientific observations and pictures of 
the planet.The project is currently [1974] an- 
ticipating a modest underrun at completion. 
So MVM ’73 more than met its original per- 
formance objectives and, in addition, served 
to work out management approaches and 
techniques to control costs. 
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Performance Measurement: A Tool For 
Program Control 

by Nancy Abell 


The NASA program and project managers of 
the 1990s will continue to work in the envi- 
ronment of constrained resources in terms of 
reduced budgets, limited staffing and tight 
schedules. In a speech to the Explorers Club 
in January 1989, former NASA Administra- 
tor James Fletcher stated: 

The funds being requested do not permit 
us the luxury of backups, of alternatives, of 
programmatic robustness. Virtually every 
element of the program is being pursued 
on a success schedule — and we know in 
advance that there will be unforeseen tech- 
nical problems to solve and dilemmas to 
face which will require internal adjust- 
ments and constraints. 

In this environment there are focused efforts 
to improve program and project manage- 
ment. One potentially powerful tool avail- 
able to the project manager which has been 
used successfully in many government agen- 
cies is performance measurement. 

Performance measurement is a management 
tool for planning, monitoring and controlling 
all aspects of program and project 
management — cost, schedule and technical 
requirements. It is a means (concept and ap- 
proach) to a desired end (effective program 
planning and control). To reach the desired 
end, however, performance measurement 
must be applied and used appropriately, with 
full knowledge and recognition of its power 
and of its limitations — what it can and can- 
not do for the project manager. 

Performance measurement is not a new con- 
cept to the government or to the aerospace 
industry. It has its origins in the Department 
of Defense (DoD) programs of the 1960s. In- 
terest and application of the performance 
measurement concept spread to other gov- 


ernment agencies in the 1970s and 1980s. 
Today performance measurement is being 
applied to major programs of the DoD, Na- 
tional Security Agency, Department of Ener- 
gy, Federal Aviation Administration and 
NASA. Performance measurement is widely 
endorsed as a valid approach to controlling 
contract performance. 

The Goddard Space Flight Center (GSFC) 
has been implementing performance mea- 
surement system (PMS) requirements since 
1983 on major research and development 
(R&D) contracts with a price of $25 million 
or more and a period of performance longer 
than one year. GSFC’s PMS policy was estab- 
lished by the Center Director to provide for 
consistent application on all major Center ac- 
quisitions. Use of performance measurement 
is also encouraged on R&D contracts in the 
$10-25 million range, but applied on a case- 
by-case basis. GSFC currently has 12 con- 
tracts in various project phases that have 
PMS requirements. With the large number 
of major independent spacecraft and instru- 
ment development contracts at GSFC, such 
as the various meteorological spacecraft and 
instruments of the Geostationary Operation- 
al Environmental Satellite and Television 
and Infrared Observational Satellite pro- 
grams, we have had the opportunity to con- 
tinually improve our implementation of PMS 
through a “lessons learned” approach. Some 
of the more effective PMS applications have 
been on the Gamma Ray Observatory and 
the Tracking and Data Relay Satellite Sys- 
tem spacecraft contracts. 

What is the potential of this management 
tool? What does performance measurement 
do that a traditional plan vs. actual tech- 
nique cannot do? Performance measurement 
provides an improvement over the customary 
comparison of how much money was spent 
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(actual cost) vs. how much was planned to be nique compared to those available from a 

spent based on a schedule of activities (work performance measurement system may serve 

planned). This commonly used plan vs. actu- to more clearly illustrate the concept. A hy- 

al comparison does not allow one to know pothetical spacecraft program is expected to 

from the numerical data if the actual cost in- take five years to build at a cost of $500 mil- 

curred was for work intended to be done. lion. Figure 1 shows the traditional plan vs. 

actual technique. If “time now” is the com- 
With performance measurement, actual pletion of year 2, the graph indicates that we 

work progress (work done, also known as had planned to spend $250 million. The actu- 

earned value) is quantified by an objective al cost (i.e., time card charges, material ex- 
measure of how much work has been accom- penses, etc.) reported to the government is 

plished on the program. This added dimen- $200 million, 

sion of a quantitative assessment of work ac- 
complished allows for comparisons to be What can a project manager conclude from 

made between the value of work that was this information? Is it possible to determine 

done vs. the work that was planned to be if this program is overrunning or underrun- 

done (schedule variance). It also allows for a ning? With this limited information avail- 

comparison of the actual cost of work that able, a project manager may assume that the 

was done vs. the planned value of the work contract is underrunning and would have no 

that was done (cost variance). This analysis basis to question the assumption that this 

then provides for early identification and program will underrun at completion. At a 

quantification of cost and schedule problems. minimum it currently appears that the $500 

A graphic depiction of the data available million funding estimate is adquate to com- 

from the traditional plan vs. actual tech- plete this effort. 



Figure 1. Traditional Plan vs. Actual Technique 
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In Figure 2 an additional data point has been 
added to the same hypothetical spacecraft 
program. The contractor has assessed the 
value of the work accomplished (or earned 
value) to date. This new information reveals 
that of the $250 million of work planned to be 
done to date, only $150 million has been 
done. Some work that was planned to be done 
has not been done and is reflected as a $100 
million schedule variance. Also, the $150 
million worth of work done can be compared 
with the actual cost of $200 million. 

This comparison shows the planned value of 
the work vs. the actual cost of that same 
piece of work. Now the project manager can 
see that this program is actually overrun- 
ning by $50 million to date. We now have 
enough data to question the validity of the 
$500 million funding estimate for completion 
of this effort. We can begin to see that this 
program is headed for an overrun of costs at 


completion along with potential schedule 
slippage. 

As a result, the project manager having the 
PMS data available in Figure 2 is better able 
to estimate early the total costs and projected 
period of performance of this program, there- 
fore avoiding a surprise overrun much later 
in the program. If the data yield a “doom and 
gloom” assessment, there is opportunity to 
make decisions early to avoid an approach 
that is too costly or that takes too long. The 
basic objective of performance measurement 
systems is to provide a suitable basis for re- 
sponsible decision-making by both the con- 
tractor and the government management by 
ensuring that (1) the contractor is using ef- 
fective internal cost and schedule manage- 
ment control systems, and (2) the govern- 
ment can rely on valid, timely and auditable 
data to be produced by those systems to de- 
termine program status. 


$IN 

MILLIONS TIME 

NOW BUDGET 



Figure 2. Performance Measurement Technique 
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Unfortunately, there has not been a consis- 
tent experience within the Agency regarding 
PMS implementation. Personnel at various 
NASA Centers and in the aerospace industry 
believe that while some NASA applications 
of PMS have been successful and effective, 
other attempts to use PMS as a management 
tool have actually been counterproductive. In 
some instances, performance measurement 
systems have not always provided accurate 
reporting of cost and schedule status, and 
there are differing opinions about why PMS 
did not work in these instances. The most 
prevalent of these is that in the NASA envi- 
ronment and culture, a disciplined approach 
to program management is not appropriate 
or applicable. 

While it is healthy to question the worth and 
applicability of PMS for NASA programs, it 
is also beneficial to explore some of the com- 
mon sense features of PMS that have proven 
effective in controlling project costs and 
schedules in many government agencies for 
the past 22 years. 

Some Basic Principles 

Performance measurement can work for you 

if you apply some basic principles. 

• Plan the entire contractual effort. It is es- 
sential to plan the work for the entire pe- 
riod of performance. Near-term work is 
planned in detail while future work can 
be planned at a summary level. Failure to 
recognize all of the work to be done makes 
it impossible to properly allocate re- 
sources. Programs could consume too 
many of the resources on the near-term 
work and not leave enough to do the work 
downstream. 

• Maintain baseline integrity. The measure- 
ment of actual conditions against a disci- 
plined or controlled plan reveals perfor- 
mance trends that can help to predict fu- 
ture conditions and to determine a future 
course of action. 


• Determine accomplishment at the level at 
which the work is performed. Who can 
better assess the work that has been done 
and the work remaining to be done than 
the manager responsible for performing 
the work? 

• Measure accomplishment objectively. The 
most valuable status assessment of a 
piece of work is based on pre-defined 
milestones as opposed to personal feelings 
and prejudices lacking reality or sub- 
stance. 

• Summarize for higher levels of manage- 
ment. While accomplishment is assessed 
at a relatively low level, summary report- 
ing to higher levels of management, 
where resources are made available, is 
also essential for control. 

• Analyze variances and forecast impact. 
Variances are simply indications that ac- 
tual conditions are different from the 
original assumptions, and variances may 
indicate the existence of current or poten- 
tial problems. Analysis of the variances 
allows management to correct problems 
or to redirect efforts to avoid potential 
problems, as well as to project cost at com- 
pletion. 

In summary, the concept of performance 
measurement is good, common sense pro- 
gram management that NASA project man- 
agers have always practiced, but perhaps not 
in a formal way. 

Specifying Customer Requirements 
NASA authority for performance measure- 
ment is based on the agency requirement 
specified in NASA Management Instruction 
9501.1 “NASA Contractor Financial Man- 
agement Reporting System” and NASA 
Handbook 9501. 2B Procedures for Contractor 
Reporting of Correlated Cost and Perfor- 
mance Data. The NASA Form 533P (where 
“P” represents performance) has been used 
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by contractors to report performance data to 
NASA, unless the contractor has another for- 
mat that serves as the equivalent. The 533P 
is essentially a minimum NASA require- 
ment for data reporting purposes only. It 
does not require that an identifiable system 
or set of subsystems support the data. As the 
contractors are free to generate data in any 
way they desire, there is the high potential 
for invalid or misleading data if this is the 
only requirement placed on a contractor re- 
lated to performance measurement. Without 
a system requirement for visibility and con- 
trol of the baseline, for objectivity in measur- 
ing accomplishment, or for discipline in fore- 
casting estimates to completion, performance 
measurement may not yield valuable infor- 
mation. While data can be reported on a 
533P, a more disciplined approach to the 
management system is needed to identify 
some rules for performance measurement 
systems. These rules are known within the 
government and aerospace industry as the 
“criteria.” 

The performance measurement criteria do 
not identify a specific management control 
system to be applied to a program; rather, 
they represent a set of standards against 
which to measure the acceptability of a con- 
tractor’s cost and schedule control system. 
There is, in fact, a variety of equally effective 
ways for contractors to meet the criteria re- 
quirements. The criteria allow a company to 
organize in any way that suits the company’s 
philosophy and style. The criteria also allow 
a company to develop any desired policies, 
procedures or methods that meet the require- 
ments. The criteria address the age-old ques- 
tions of any project manager: What work is to 
be done? Who will do it? When is it going to 
be done? How much will it cost? Where is the 
program heading? What has changed? 

The contractors address these questions 
through their management systems’ inte- 
grated set of subsystems. These are subsys- 
tems that would be required to manage a pro- 


gram whether or not a performance measure- 
ment requirement was imposed. Perfor- 
mance measurement criteria simply require 
that a more disciplined approach be applied 
to each subsystem. The PMS subsystems are 
(1) work authorization, (2) budgeting, (3) 
scheduling, (4) data accumulation, (5) vari- 
ance analysis and estimate at completion, (6) 
subcontract and material control and ac- 
countability, (7) indirect expense manage- 
ment, and (8) change baseline control. PMS, 
then, does not address just the accounting 
system, but rather it addresses the integrat- 
ed set of subsystems that constitute all ele- 
ments of program planning and control. 

A Good Management System 
The key to the power of performance mea- 
surement is that performance measurement 
data are only as valid as the management 
system that provides them. If a contractor op- 
erates a sound internal management system, 
the customer should be able to extract sum- 
mary data from that system that reflect pro- 
ject status. To have a valid management sys- 
tem applied to NASA work in contractor 
plants, several conditions need to be met. 

First, a management commitment from the 
top down is required — all levels of manage- 
ment support are essential. It is not enough 
to have project financial or resources support 
personnel discussing PMS with the contrac- 
tor. The involvement of technical personnel 
is critical. PMS involves all aspects of pro- 
gram management and needs to be viewed in 
this way by NASA project and functional 
management personnel to be effective. 

Second, management system discipline must 
be stressed and required. While it may be de- 
sirable to maintain a spirit of cooperation 
and non-adversarial relations with our con- 
tractors, PMS is not of any value without a 
disciplined approach to management. With- 
out a requirement for the contractor to main- 
tain a baseline, to apply objective techniques 
for performance measurement, or to reliably 
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forecast the cost to completion, there can be 
no confidence in the value of the data that 
the management system generates and that 
the contractor reports to NASA on a monthly 
basis. 

Third, use of data generated by the PMS is es- 
sential. A few simple mathematical formulas 
and computations yield very revealing infor- 
mation about the project status and potential 
future of the program. Use of data serves to 
facilitate communications internally and be- 
tween NASA and the contractor. 

Fourth, corrective action needs to be taken 
when problems are identified. A manage- 
ment system supplies data points, not solu- 
tions. It provides visibility into cost, schedule 
and technical status. A system, however, 
does not manage the project, people do. A sys- 
tem cannot eliminate schedule slippages or 
stop overruns, but it can help the project 
manager to understand the potential impact 
if trends are allowed to continue without 
mid-course correction. 

Fifth, an in-plant review of the contractor’s 
management system applied to a program 
and conducted by a NASA team of interested 
and knowledgeable technical and resources 
personnel is critical. The NASA personnel 
gain invaluable knowledge of the policies, 
methods and procedures used by the contrac- 
tor to generate monthly status reports. By 
understanding the source of the data, we can 
calibrate the validity of our monthly custom- 
er reports and require the contractor to re- 
vise procedures that do not produce valid 
data. 

PMS is not intended to replace traditional 
management tools — it should enhance them. 
Day-to-day program management is essen- 
tial. In fact, if managers are relying solely on 
performance measurement data generated at 
month-end, they will be learning of problem 
situations much too late to be effective. Peri- 
odic status reviews, “kicking the tires,” and 


routine communication internal to the con- 
tractor and between the contractor and gov- 
ernment managers are critical in managing 
a program. PMS may identify a new problem, 
but in most cases, it allows quantification of 
a known problem through all elements of the 
work breakdown structure and through the 
functional organizations to provide a basis 
for improved management decisions. 

Cost Effectiveness 

In times of constrained resources it is reason- 
able for managers to question the cost effec- 
tiveness of PMS. What are the benefits and 
associated costs? The question is difficult to 
answer, however, since both the benefits and 
costs are nearly impossible to quantify. 

PMS results in a better controlled project 
with improved communication, both inter- 
nally and with the customer. To quantify the 
benefits is to ask, “What is the value of good 
management?” It is not evident how a cost 
savings (or cost avoidance), a shortened 
schedule, or improved technical performance 
through corrective action can be clearly asso- 
ciated with results or a specific cost. 

The costs of PMS have also defied quantifica- 
tion for 22 years. The PMS-unique costs on 
the total contract cannot be separately iden- 
tified from the management costs that would 
be incurred in any case. They are not rou- 
tinely collected by contractors, nor is it con- 
sidered practical to do so. This was illustrat- 
ed in a 1987 survey of GSFC contractors who 
had implemented a PMS requirement. In the 
survey, some contractors suggested that the 
costs of PMS beyond the usual management 
costs may be expressed as a percentage rang- 
ing from 2 percent to 6 percent of total con- 
tract costs. In each case, however, the con- 
tractor could not substantiate the percent- 
age. It was someone’s “non-scientific esti- 
mate,” as stated by one contractor. Surveys 
conducted by the DoD show that there is no 
correlation between the cost of PMS and the 
contract costs. 
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This is not to say that there cannot be costs 
associated with PMS requirements. In fact, 
the cost of implementing PMS is in direct 
proportion to the quality of the existing man- 
agement system. The poorer the state of the 
contractor’s system, the greater the need for 
improvement and the more it will cost to im- 
prove. Contractors who maintain discipline 
in their systems would incur very low costs to 
implement PMS on subsequent contracts. If 
the same contractors did not maintain their 
systems, over time the cost to implement 
PMS on future contracts would be greater as 
the need for improvement became greater. 

Further, if there is not an existing integrated 
cost and schedule management system, the 
contractor will certainly incur costs to devel- 
op one. GSFC experience, however, has been 
that contractors awarded major development 
procurements that contain PMS require- 
ments are contractors who already have 
operational PMS systems as a result of their 
dealings with DoD. Costs of PMS have been- 
minimal compared to the significantly great- 
er value added. 


There is one additional factor to consider in a 
discussion of the costs of PMS. Typical points 
of contention between the government and 
industry concerning PMS implementation 
include the levels of detail identified for 
management and reporting, and the vari- 
ance analysis thresholds identified for cus- 
tomer reporting. It is possible to avoid incur- 
ring unnecessary cost to the government and 
frustration for the contractor by not request- 
ing reports that no one reads or uses, or “nice 
to have” items or analyses. 

In summary, with the focus on efforts to im- 
prove program and project management, 
PMS is a potentially valuable tool. Like any 
tool, however, it is only as valuable as the 
user chooses to make it. Implemented proper- 
ly, PMS can ensure the generation of valid 
cost and schedule performance data to ease 
the manager’s decision-making process, 
which can result in more effective program 
planning and control. 
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Program Control for Mission Success 


by G. W. Longanecker 

My first premise is that in order to exercise 
program control, you must have a controlla- 
ble program, which is one that has been prop- 
erly scoped technically, realistically sched- 
uled, and adequately budgeted. 

The first step in scoping a program is obtain- 
ing a set of minimum performance require- 
ments to meet the mission objectives. I know 
that this is a difficult task, because your cus- 
tomer is intent on achieving the maximum 
possible performance. However, my recom- 
mendation is to get an agreement with your 
customer on the minimum requirements, 
and then set the specifications to achieve a 
reasonably increased level of performance. 
This will allow for possible descoping actions 
later in the program, should the need arise. 
Since our programs nearly always involve 
state-of-the-art technology, and with today’s 
emphasis on resource control, a good descop- 
ing plan developed early in the program is 
important to have in your back pocket. 

The other two ingredients of a controllable 
program are schedule and cost. The two are 
very much interdependent and must be bal- 
anced with the degree of risk deemed appro- 
priate for the program. There has been a lot 
of rhetoric on the subject of risk, especially in 
recent years. However, in my 30 years with 
the agency, I really didn’t see much risk- 
taking, even with the unmanned scientific 
and applications satellite programs. Risk is 
extremely difficult to quantify, especially 
when you’re dealing with single satellite pro- 
grams. How do you explain a risk trade-off to 
a group of space physicists who are commit- 
ting possibly half of their professional ca- 
reers to a single satellite mission? 

My consummate goal was always mission 
success. What this really boils down to is that 
you need to have adequate schedule slack 


and budget contingency to solve the inevita- 
ble problems that will confront you along the 
way. Headquarters must hold sufficient re- 
serves to cover any changes in scope. This is 
important enough to reiterate. The project 
manager at the field Center budgets and con- 
trols reserves for problem solving; the pro- 
gram manager at Headquarters budgets and 
controls reserves for scope changes. The last 
line of defense is to descope the program. 

As I said earlier, if you have set your specifi- 
cations with some margin over the minimum 
goals, you should have some room to descope 
and still meet mission objectives. The real 
challenge for a manager is that you probably 
will have to make some descoping decisions 
during the development phase so that you 
have some remaining contingency for the 
test and evaluation phase, mission oper- 
ations, data collection and data processing. 

Properly scoping a program requires that 
sufficient studies be performed during the 
definition phase. As a rule of thumb, four to 
eight percent of the expected total run-out 
cost of a program should be spent through 
Phase B. In my experience, NASA is notori- 
ous for skimping on definition-phase fun- 
ding. When you skimp during Phase A and 
Phase B, you have an open invitation to per- 
formance, schedule and budget problems 
during Phases C and D. As part of the pro- 
curement planning process, you will develop 
in-house a “should-cost” estimate for the pro- 
gram. Your budget requests will be based on 
this “should-cost” figure plus contingency. 
Because of competition, you will most likely 
negotiate a contract for less than the 
“should -cost” estimate. The difference should 
not be considered part of your contingency 
for problem solving, but rather it represents 
the additional funds required to realistically 
perform the prescribed effort without prob- 
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lems. Occasionally a contractor will propose 
a scheme that should save some money, but 
again my experience has been that you 
should pay attention to your “should-cost” es- 
timate. 

Beyond the programmatic obstacles to a con- 
trollable program, the single biggest hard- 
ware obstacle in my experience has been 
piece parts. I can’t remember a single pro- 
gram (and I’ve launched 21 satellites) where 
we didn’t have problems with piece parts. 
We’d design a circuit, breadboard it, test it 
and then find that we couldn’t get flight- 
qualified versions of the parts. We also suf- 
fered from being a small-volume user of piece 
parts since most of our programs involved a 
single satellite. The only advice I can offer is 
to use standard parts as much as possible in 
your designs, order your parts as early as 
possible in the program, and look for second- 
source suppliers for your critical parts. Even 
after doing all of the above, the odds are that 
you will have piece part delivery problems. 

As for program control, there are many good 
techniques and tools. Everything starts with 
a good work breakdown structure (WBS). 
You will have developed one during the defi- 
nition phase and for the Phase C and D pro- 
curement package and, subsequent to con- 
tract award, will agree to the WBS with your 
prime contractor. The WBS is the basis for 
your schedule projection and budget esti- 
mate. It must have sufficient granularity to 
identify the critical elements or building 
blocks of the program. 

Your schedule must have slack identified at 
critical points in the program. It is not suffi- 
cient to carry all the slack in the period just 
before the launch readiness date. This is es- 
pecially true when you’re dealing with inter- 
governmental or international partners in a 
cooperative program. In most cases you’ll 
find that the cooperating agencies have even 
less flexibility to deal with schedule and bud- 
get changes than we do in NASA. Once es- 


tablished, the schedules can be tracked by 
any number of computer-generated systems. 
Critical paths are easily identified and 
tracked. However, I advise you not to rely 
solely on the automated schedule systems. 
I’ve always found it useful to prepare a few 
charts on critical elements that I could up- 
date manually to look for schedule trends. 
My favorite is one that tracked on a monthly 
basis, for a few selected milestones, the cur- 
rently planned date versus the originally 
scheduled date (Figure 1). 

I would frequently find that I could apply the 
slope of the trend for intermediate miles- 
tones to forecast* the most probable comple- 
tion date for a downstream event, even 
though the contractor continued to forecast 
the original event date. I found it easier to 
look at my few graphs than to study the 
computer-generated charts covering the 
walls of the “war room.” You have to keep a 
perspective on the big picture. 

The final element of program control that I 
wish to discuss is a performance measure- 
ment system (PMS), or earned value system, 
which allows you to track progress versus ex- 
pended resources compared to your plan. Es- 
sentially all major contractors have a PMS 
that they use for their programs. The key 
word here is “use.” Having a PMS in your 
contract is a useless exercise if the contractor 
is not actually using the system to help man- 
age the program. Accordingly, you should 
adopt the system your contractor is familiar 
with, rather than insist on a similar but dif- 
ferent system. Due to the nature of our busi- 
ness, changes to the program baseline are to 
be expected. Obviously, such changes should 
be kept to the absolute minimum, but when 
it’s unavoidable, any significant change 
must be quickly incorporated into the PMS. 

Reporting earned value against an outdated 
plan is useless at best. It can be worse than 
useless if someone believes data that is blind- 
ly cranked out, based on an outdated plan. If 
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SCHEDULE DATE 



Figure 1. Sample Trend Chart 

the data is current, a PMS can help you de- As with automated scheduling systems, PMS 

tect the trouble spots sooner and, therefore, is not a panacea for the managers. You have 

direct your problem-solving energies more to keep track of the big picture, and above 
efficiently. all, use good old common sense. 
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The MSFC Program Control Development Program 


It is the policy of the Marshall Space Flight 
Center (MSFC) that employees be given the 
opportunity to develop their individual skills 
and realize their full potential consistent 
with their selected career path and with the 
overall Center’s needs and objectives. 

The MSFC Program Control Development 
Program has been designed to assist indi- 
viduals who have selected Program Control 
(Figure 1) or Program Analyst Program Con- 
trol (Figure 2) as a career path to achieve 
their ultimate career goals. Individuals se- 
lected to participate in the MSFC Program 
Control Development Program will be pro- 
vided with development training in the var- 
ious Program Control functional areas iden- 
tified in the NASA Program Control Model 
(Figure 3). 

The MSFC Program Control Development 
Program should be mutually beneficial to the 
individual and to MSFC. Each individual 
will be afforded the opportunity for pursuing 
his or her career goals while simultaneously 
providing the Center with a source of knowl- 
edgeable and well-trained people for strate- 
gic positions within the Program Control dis- 
cipline. The purpose of the MSFC Program 
Control Development Program is to develop 
individual skills in the various Program 
Control functions by on-the-job and class- 
room instructional training on the various 
systems, tools, techniques and processes uti- 
lized in these areas. 

The Program offers a systematic approach 
for individual development by: 

• Allowing for rotational job assignments 
to obtain on-the-job training in the var- 
ious Program Control functional areas. 


• Providing classroom instructional train- 
ing on various program control systems, 
processes and procedures to assist in the 
development of skills required for the 
Program Control functional areas. 

• Encouraging continuation of the individ- 
ual’s formal education as a means of self- 
improvement. 

The MSFC Program Control Development 
Program is made available to individuals in 
the AST classification, as well as individuals 
in the Program Analyst classification who 
have selected Program Control as a career 
path, and who have been selected for partici- 
pation in this program. Applicants may be 
nominated by the organizations or self- 
nominated. 

Selection Process 

A committee is appointed by the administra- 
tive Operations Office to review and select 
applications for the MSFC Program Control 
Development Program. The committee is 
comprised of membership from the Adminis- 
trative Operations office, Comptroller’s office 
and the Program or Project Office. 

The means to be used for publicizing vacan- 
cies for this program may be reassignment 
announcements, The Personnel Perspective, 
The Marshall Star or other media. The oppor- 
tunities will open to GS-11, -12 and -13 AST 
applicants, with potential progression to the 
GS-13 level, if selection was at a lower level, 
based upon successful completion of the pro- 
gram. They will open to GS-11 and GS-12 
Program Analyst applicants, with potential 
progression to the GS-12 level, if selection 
was at a lower level, based upon successful 
completion of the program. 
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GS 9-1 1 GS/GM 11-13 GS/GM 13-14 GS/GM 14-15 GM15-SES 


Broadening Activities: Participation in the numerous Program Control related training courses and pursuing a formal or 
continuing educational program at a selected university are encouraged as a means for self-improvement that should 
enhance an employee's career progression opportunities. 


Figure 1. Example of AST Program Control Career Progression 


Selection is based on the individual’s applica- 
tion and personal interviews. Notification of 
selection and final appointment to the pro- 
gram will be made by the Administrative 
Operations Office, following a review by the 
Director of Administrative Operations and 
the Comptroller. 

Individuals selected for this program are as- 
signed to the MSFC Comptroller’s Office for 
a two-year developmental period. During 
this period, individuals will be temporarily 
assigned on a rotational basis to offices iden- 
tified for specific on-the-job training for the 
indicated periods. To the extent possible, re- 


lated classroom instructional training for all 
Program Control functional areas will be of- 
fered during this two-year period. 

Standard Individual Development Plans will 
be prepared for each individual to schedule: 

• Rotational assignments, in order to mini- 
mize any impact on the organization pro- 
viding the training. 

• Classroom instructional training courses 
identified in this program, as well as oth- 
er pertinent courses that might become 
available. 
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Figure 2. Example of Program Analyst Program Control Career Progression 


• Formal or continuing educational pro- 
grams selected by the individual for self- 
improvement. 

Administrative Responsibilities 

The responsibility for administration of the 

MSFC Program Control Program is: 

• The Comptroller’s Office will be responsi- 
ble for the administration of the program; 
the technical and programmatic content 
of the program; preparation of Standard 
Individual Development Plans, including 
planning and scheduling rotational job 


assignments, classroom instructional 
training and formal or continuing educa- 
tional programs; and normal personnel 
supervisory responsibility for the two- 
year development period. 

• Administrative Operations will be re- 
sponsible for preparing and issuing An- 
nouncements for the MSFC Program Con- 
trol Development Program and the indi- 
vidual selection process; providing class- 
room instructional training; and counsel- 
ing individuals on formal educational op- 
portunities. 
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Figure 3. Program Control Functions 


• Organizational elements providing on- 
the-job training will be responsible for 
providing the training set forth in the 
Standard Individual Development Plans 
for their particular area of responsibility 
and assisting with arrangements for 
classroom instructional training as ap- 
propriate. A mentor will be designated by 
the organizational element supervisor to 
assure that required on-the-job training 
is accomplished to the extent possible. 

The participating individual, in concert with 
the mentor in the participating organization, 
will be responsible for completing (to the ex- 
tent possible) on-the-job training require- 
ments set forth in the Standard Individual 
Development Plans, planning and attending 
classroom instructional training, and other 
self-improvements such as formal or continu- 
ing educational programs. 

Rotational Job Assignment. A major fea- 
ture of the MSFC Program Control Develop- 
ment Program is that of rotational job as- 
signments. Table 1 presents typical rotation- 
al job assignments to which participants will 
be assigned over the two-year training pro- 
gram. 


Table 1. Examples of Rotational 
Job Assignments 


Assignment 

Months 

Comptroller’s Office 

4 

NASA Headquarters 

2 

Procurement Office 

2 

Project/Program Office 

5 

Engineering Cost Group 

3 

S&E Resources Management 

3 

Configuration Management 

3 

Resident Office 

2 

Total 

24 


These job assignments will allow the individ- 
ual to receive on-the-job training to satisfy 
the training requirements set forth below. 

Training Requirements. Table 2 lists typi- 
cal training requirements for each job as- 
signment. This on-the-job training, coupled 
with classroom instructional training con- 
ducted during the two-year period, will pro- 
vide the individual with the training needed 
for each Program Control area. Additionally, 
it should prepare the individual for job oppor- 
tunities of greater responsibility on his or 
her career development path. 
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Table 2. Typical Training Requirements for Each Job Assignment 


NASA HEADQUARTERS 

Program Office 

NASA Comptroller 

- Organization Structure 

-Organization Structure 

- Roles and Responsibilities 

-Roles and Responsibilities 

-Working Relationships 

-Working Relationship With Program Office 

Field Center Program Offices 

- Independent Assessments 

NASA Comptroller 
Other NASA HQ Offices 

- Non-Advocacy Review Assessments 

- Interactions With OMB/Congress 
-Federal Budget Process 

- NASA Budget Process 


COMPTROLLER'S OFFICE 


POP Process 

Institutional Operating Plan 
Research and Program Management 
Budget 

Manpower Planning 
-918 Report 

- Manpower Management Info System 
Resources Management Info System 
Federal Budget Process/NASA Budget 
Process 

Receipt, Allocation and Control of Funds 

- 504 and 506 Reports 

- Resources Authority Plan 
NASA Organization 
MSFC Organization 
Program Control Overview 


Contractor Reporting of Correlated Cost 
and Performance Data (533 Reports) 
Performance Measurement System 
NASA Financial Management System 
NASA/ MSFC Accounting System 
Comptroller Office 

- Organization Structure 

- Roles & Responsibilities 

- Financial Management Operations 
-Working Relationships 
Program and Project Office 

NASA Comptroller 
NASA Program Offices 
Construction of Facilities Budget Process 
Program Support Requirements 
Project Planning Process 


Project Plans and Requirements 

- Project Plan 

- Management Plan 

- Implementation Plans 

- Requirements and Specifications 
Schedules 

- Logic Networks 

- Project Schedules 
Master 

- Supporting 

- Status/Reporting/Analysis 
Work Breakdown Structure 

-Project WBS 
-Contract WBS 


PROJECT/PROGRAM OFFICE 

Budgeting 
-Project Budgets 

- Contractor Budgets 

- 533 Reports 

- Performance Measurement 

- Cost Analysis 
Project/Program Office 

- Organization Structure 

- Roles & Responsibilities 
-Working Relationships 

S&E 

Procurement 
Project/Program Office 


NASA HQ Project/Program Office 

Comptroller 

Other Centers 

Others 

Project/Program Reviews 
Project Reporting 
Data and Information System 
- Data Information System 
-Data Base 
-Modules 
-Analysis 

Independent Analysis 


PROCUREMENT OFFICE 


Contract Management 

- Procurement Planning 
-Solicitation Process 

(Including SEB Process) 

- Contract Negotiation 

- Contract Administration 
-DCMC Role and 

Responsibilities 
-Award Fee Process 

- Change Assessment 


Pricing 

-Contract Pricing 
- Rates & Factors 
-Inflation Factors 
-Overhead/G&A Rates 
-DCAA/DCMC Role and 
Responsibilities 
-Forward Pricing 
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Table 2. Typical Training Requirements for Each Job Assignment (cont.) 


S&E RESOURCES MANAGEMENT OFFICE 

Research and Technology Operating OAET Budget Process 

Plans S&E Directorate 

Institutional Planning Budgeting Process -Organization Structure 

Budget Execution - Roles and Responsibilities 

- Source of Funding Relationships to Program/Project 

Manpower Planning/Control Offices 

- Skills Analysis ~ Program Support Agreements 

Purchase Requests " Pro 9 ram Su PP ort ^^ments 

Facilities and Equipment Requirements 

ENGINEERING COST GROUP 

Cost Estimating Techniques 
- Para metric 
-Analogy 
-Bottom up 

Cost Estimating Relationships 
-Cost Modeling 
-Cost Risk Assessment 
-Economic Benefits Analysis 
-Redstar Data Base 
-Sensitivity Analysis 
-Trade Studies 

CONFIGURATION MANAGEMENT DIVISION 


Configuration Management Division 

Change Control Boards 


-Organization Structure 

-Organization Structure 

- Roles & Responsibilities 

- Membership 


- Working Relationships 

- Operation 


Program/Project Offices 

Tracking and Accounting System 

S&E Laboratories 

Verification 


Safety and Mission Assurance 

Documentation and Data 

Configuration Management System 

Management System 


-Identification 

-Data Requirements 


- Control 

- Data List 


-Accounting 

- Data Procurement Document 

-Verification 

- Data Classification 


Configuration Mgmt. Requirements 

Documentation Tree 


Change Process 

Specification Tree 


- Engineering Change Request 

- Program 


- Engineering Change Proposal 

- Project 


- Change Control Board Directive 

-Contract End Item 


-Change Flow 

-Critical Procurements 

-Change Integration 

Interface Control 


Baseline Reviews 

-Requirements 


- Preliminary Requirements Review 

- Documents 


- Preliminary Design Review 

Document Processing 


-Critical Design Review 

- Electronic Processing 


- Design Certification Review 

- Archiving 


-Configuration Inspection 
- Acceptance Review 

- Repository 



RESIDENT OFFICE 


Resident Office 
-Organization Structure 

- Roles & Responsibilities 
-Working Relationships 

Program/Project Office Contractor 
Defense Contract Management 
Command (DCMC) 

Defense Control Audit Agency 
(DCAA) 

Contractor 

- Organization Structure and 
Responsibilities 


Contractor (cont.) 

- Internal Planning & Control System 
-Work Authorization System 

- Interdivisional Work 
Authorizations 

- Subcontract Management 

- Internal Functional 
Organizational Relationships 

- Contractor Overhead/Burden 

- Requirements 
-Approvals 

- Budget Allocations/Control 


Contractor (cont.) 

- General and Administrative 
Expenses 

- Preparation of Contractually 
Required Data 

-Direct Budgets 

- Requirement Determination 

- Direct Budget Allocation 

- Cost Collection 

- Statusing 

- Cost Reporting 
-DCMC 
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Training Courses. Table 3 identifies nu- 
merous training courses available to MSFC 
employees. The training courses are extract- 
ed from the latest issue of the MSFC Train- 
ing Course Catalog and have been grouped 
into the various Program Control functional 
areas to provide the individual with informa- 
tion on the many related course opportuni- 
ties available. The list is representative and 
not all inclusive. The designation “N/A” 
means that the information was not avail- 
able at time of printing. Some of the courses 
listed may no longer be available or may 
have changed. Some new training courses 
may become available that are not listed. For 
these reasons, it is important that the par- 
ticipant and his or her supervisor or mentor 
consult with the MSFC Training Branch in 
planning individual training needs. 

Because of the large number of training 
courses available in certain functional areas, 
it will be necessary that the individual work 
with his or her supervisor and mentor in se- 
lecting the minimum essential courses need- 
ed to successfully complete the program. The 
Program Control Overview courseis consid- 
ered mandatory for this program. It will be 
presented locally on an ad hoc basis. 

Continuous Self-Improvement Program 
Individuals selected for the Program Control 
Development Program are encouraged to 
participate in a formal or continuing educa- 
tional program. There are many opportuni- 
ties for individuals to seek self-improvement 
through various educational programs relat- 
ing to the Program Control career path: 

• Graduate programs at a selected univer- 
sity: 

- Master of management at the Universi- 
ty of Alabama at Huntsville (UAH), 
which requires an undergraduate de- 
gree in engineering. 


- Master of Science in engineering (with 
options in industrial engineering, sys- 
tems engineering, etc.) and Master of 
Science in operations research at UAH. 

- Master in engineering or a Master of 
business administration in Auburn 
University’s “Outreach Program.” 

- Master of Science in industrial man- 
agement at the University of Tennessee 
Space Institute. 

- Master of business administration at 
Alabama A&M in cooperation with 
Pennsylvania State University. The 
MBA curriculum is specifically de- 
signed to give the student an opportuni- 
ty to obtain a degree in business, re- 
gardless of the field in which he or she 
majored at the undergraduate level. 

- Masters degrees in a number of non- 
engineering areas at the Florida Insti- 
tute of Technology. 

• Continuing education programs: 

- UAH Division of Continuing Education 
offers numerous educational opportuni- 
ties designed to enhance professional 
and personal development. Continuing 
education credit courses enable stu- 
dents to pursue an undergraduate or 
graduate degree. Continuing education 
non-credit short courses and certificate 
programs provide quality training and 
education for professional and personal 
development. Especially noteworthy is 
the Project Management Certificate 
Program, a non-degree program requir- 
ing about 81 hours of classroom work 
over a period of about six months. A 
certificate from UAH or the Project 
Management Institute is awarded upon 
successful completion of the program. 
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The details of the above and other education- 
al programs are available in the Personnel 
Development Division, Administrative Oper- 
ations Office, MSFC. Individuals electing 
Program Control as a career development 
field are encouraged to give serious consider- 
ation to the many educational program op- 
portunities available to them. 

Upon completion of the MSFC Program Con- 
trol Development Program, individuals will 
be assigned to a permanent position in a va- 


cancy in the Program Control discipline com- 
mensurate with his or her position (title, 
grade and series) and career goals. 

The MSFC Program Control Development 
Program offers no guarantees of promotion 
or career changes, except for those contained 
in the paragraph "Selection Process" above, 
but it does help individuals develop and grow 
personally and professionally, thus enhanc- 
ing their value to the Center by improving 
their qualifications for future opportunities. 
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Table 3. MSFC Program Control Training Courses 


Program Control 



Resources Management 



Course Title 

Course No. Duratior 

Course Title 

Course No. 

Duration 


or Sponsor 

(hrs) 


or Sponsor 

(hrs) 

Program Control Overview 

NASA 

40 

Federal Budget Process 

40203 

16 

Project Management Workshop 

41549 

24 

NASA Budget Process 

N/A 

N/A 

Research and Development 



Cost/Manpower Mgmt. 



Contracting 

40772 

15 

(includes Federal 

20125 

16 

Source Evaluation Procedures 

40776 

24 

Budget Process and 



(for FED Procurements) 



NASA Budget Cycle) 



Source Evaluation Procedures 

41554 

15 

Engineering Economics 

30433 

40 

Technical Project Management 

40971 

24 

Introduction to Financial 



Types of Government Contracts 

40785 

24 

Management 

40249 

40 

Basic Planning and Analysis 

N/A 

40 

Budget Formulation 

40177 

40 

Introduction to Space Systems 

N/A 

36 

Budget Execution 

40176 

40 

Contract Changes and Terminations 40706 

40 

Cost Analysis and 
Estimating Techniques 

40531 

32 

Federal Acquisition Process 

41537 

N/A 

Cost and Price Analysis 

40192 

40 

Contractor Project Planning and 



Advanced Cost and Price 



Control System 

N/A 

N/A 

Analysis 

41526 

40 

Project Organization Structure 

N/A 

N/A 

Budget Analysis Workshop 

40174 

40 

Applied Quality Assurance 
Operations 

30964 

15 

Statistical Techniques for 
Analysis 

30406 

40 

Cost Contracting 

41532 

40 

Performance Management 



Contract Analysis and Control 
(for Project Management) 

40704 

09 

System 

N/A 

32 

Problem Solving & Decision Making 

20114 

24 




Basic Procurement 
Contract Management for 

40693 

40 




Engineers 

41522 

40 

Schedule Management 



Contract Planning 

41529 

40 

Course Title 

Course No. 

Duration 

Developing Work Statements 




or Sponsor 

(hrs) 

for R&D Contracting 

41535 

32 

PERT-CPM Workshop 

41547 

20 

Evaluating a Contractor’s 



Program Evaluation 



Performance 

41536 

40 

and Review Techniques 

40945 

24 

Fundamentals of Contract 
Administration 

40735 

40 

Program Planning 
Network Risk Analysis 

40949 

20 

Fundamentals of Program 
Management 

40935 

40 

Computer Programs: 
Artemis 

N/A 

N/A 

Government Contract Negotiation 
Techniques 

40740 

40 

Project II 
Primavera 
Timeline 
Quicknet 

N/A 

N/A 

Techniques of Negotiating 

40780 

24 



Government Contract Negotiations 

40741 

40 



Negotiations 

41546 

40 



Management Analysis and Review 

40509 

40 
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Table 3. MSFC Program Control Training Courses (cont.) 


Change Management 



Information Management 


Duration 

Course Title 

Course No. Duration 

Course Title Course No. 

or Sponsor 

(hrs) 

or Sponsor 

(hrs) 

Configuration Management 



Managing Information: 



of Software 

40930 

24 

Making Information 



Configuration Management 

UAH 

09 

Work for You 

41558 

02 

Integrated Configuration 
Management 

UAH 

40 

Workshop in Management 
Information Systems 

90383 

02 

Advanced Configuration 
Management 

Tech Trng Corp. 

24 

Presenting Data in Graphs 
Charts and Tables 

40515 

24 

Strategies & Techniques for 
Configuration Management 

Teen Trng Corp. 

16 

.Data Base Systems 

90176 

40 

Basic Configuration Mgmt./ 
Advanced Config. Mgmt. 

Acquisition Techniques for 

CDM Consultants 

32 




Configuration 
Management Practitioners 

SCITEK 

24 




Basic Data Management 

Tech Data Inc. 

16 




Data Management Course, 
Number 15934 

Navy Consld. 

jy / A 




^iv rers uTTice 





Configuration 

Inst, of Config. 





Management II 

Mgmt/Arizona 


Plans and Requirements 


Duration 

(hrs) 

Course 1-Documentation 
and Change MGMS 


24 

Course Title 

Course No. 
or Sponsor 

Course 11-Requirement 



Systems Engineering 

NASA 

36 

Specifications Review 


24 

Process 

Course Ill-Configuration 



- Systems Engineering 



Definition & Documentation 

24 

Process 



Course IV-Engineering Change 


- Mission Need Statement 


Control and Traceability 


24 

-System Requirements/ 



Course V-Change Boards 


* 24 

Specifications 



Change Administration 


24 

- Implementation Plans 



Course Vl-Organize and 



-Baseline Reviews 



Manage Requirements 



SRM & QA Process 

N/A 

N/A 

Course X-CMII for Executives 


08 

Fundamentals of Logistics 



(Generic for Public) 

Course XI-CMII for Executives 
(for Onsite) 


Management 

40490 

40 


04 
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NASA’s Attack on Costs 

by George M. Low 

If we don’t do something about the high cost 
of doing business in space, and do it soon, our 
nation’s space program is in deep trouble. We 
are on the verge of exciting new discoveries 
in space science, but we cannot follow 
through as rapidly as we should because we 
can’t afford it. We see before us many impor- 
tant space applications, but we cannot move 
out as rapidly as we should because we can’t 
afford it. Most important of all, we may lose 
our hard-won worldwide leadership in space, 
if we don’t find a way to do more for our mon- 
ey! To put it another way, we are doing less 
than we should do in space because things 
are so expensive, and because we are work- 
ing under very tight budgetary constraints. 
There is very little we can do about the 
budget — it is imposed by external forces, and 
there are many other pressures on it; but 
there is a great deal we can do about costs. 
Doing something about the high cost of doing 
business in space is today’s biggest chal- 
lenge. 

Basically, when we started out in this busi- 
ness 14 years ago, launch costs were overrid- 
ingly important. We crammed everything we 
could into each launch, and optimized our 
satellites for low weight, low volume and 
high performance. Payload costs were clearly 
secondary considerations. This resulted in 
spacecraft tailor-made for each mission, with 
its own customized subsystems and compo- 
nents. Every piece was designed to the limit, 
with low margins and low tolerances. The 
net result was that most systems could be 
used only for the purpose for which they were 
originally intended, and after use in one sat- 
ellite, were discarded in favor of new systems 
designed for the next mission. Furthermore, 
the low design tolerances required a tremen- 
dous number of tests and enormous volumes 
of paperwork to achieve absolute control. In 
other words, we were forced to perform very 
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expensive developments and then used the 
spacecraft only once or twice before starting 
on the next very expensive development. We 
have built many one-of-a-kind items, and 
have demanded the utmost in performance 
from them. Simple statistics bear this out. 
Today we have 52 operating civilian space- 
craft. By the end of next year we will launch 
35 more, for a total of 87. These 87 represent 
43 different spacecraft types — on the average 
each spacecraft is flown only twice! 

But things are different now. Payloads have 
become more complex, while launch costs 
have come down . . . This means that we 
should now optimize our payloads for low 
cost and high reliability, and not for mini- 
mum weight and maximum performance as 
we did in the past. I am convinced that if we 
do this, we can drastically reduce the cost of 
doing business in space. And this, I believe, 
is as great a technological challenge as ev- 
erything else that we have done in space. . . 
In the design phase, the following principles 
are important: 

• Don’t re-invent the wheel. Use the best 
that is available from other programs. In 
all of the commercial firms I have visited, 
“not invented here” is unheard of. All tear 
down their competitor’s product, study it, 
analyze it, cost it and make use of the best 
ideas in it, as long as they do not infringe 
on patent rights. 

• Standardize. This applies to parts, compo- 
nents, modules, subsystems and entire 
systems. (There are only two chassis for 
the entire line of Sears television sets, 
and even they contain many common 
modules.) 

• Design for low cost. Involve production 
engineers in the earliest stages of design 
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to help eliminate those things that will be 
difficult to produce. 

• Design to minimize testing and paper- 
work. Simply stated, this means: take ad- 
vantage of reduced weight and volume 
constraints and use standard parts, larger 
margins, and larger safety factors. (In 
Apollo we spent millions of dollars — on 
tests and paper — to be sure we did not ex- 
ceed the “fracture mechanics” limits on 
our pressure vessels. A few extra pounds 
in tank weights would have completely 
eliminated that problem, and the testing 
and paperwork along with it.) 

• Recognize that different systems can ac- 
cept differing degrees of risk ... A Shuttle 
life support system must be more reliable 
than a simple experiment in the Shuttle 
payload bay. The cost of a system should 
reflect the acceptance of risk in those in- 
stances where this is possible. 

• Know your costs. None of the things I 
have said has any meaning if we don’t 
know how much each element costs. The 
area of accurate cost estimating is one 
where we have a great deal to learn. Yet 
it is an area which has been developed. 

• Trade features for cost. This follows natu- 
rally from the previous item. Once we 
know how much something costs, then we 
can ask ourselves whether it is really 
worth it. Many of our so-called “require- 
ments” really aren’t that firm, and should 
be stated as “goals,” to be re-examined in 
terms of cost. 

t Pay particular attention to the few very 
high-cost items. In many designs some 
small percentage of the items amount to 
most of the costs. By knowing the costs, 
and by listing items in order of descend- 
ing costs, it becomes possible to devote a 


great deal of attention to the high-cost 
items — generally with profound results. 

In the implementation phase, I would em- 
phasize the following points: 

• Know your costs before you start. This per- 
haps is the most fundamental of all re- 
quirements. Without exception, the 
NASA programs which have been in diffi- 
culty were the ones that had insufficient 
definition at the outset. 

• Set firm cost targets. A desire for the “low- 
est possible cost” is not a good way to ap- 
proach the job. A firm and absolute cost 
ceiling should be established for each job. 

• Meet the established cost targets. Don’t 
blame cost growths above target on “ex- 
ternal forces.” Find ways to meet the tar- 
gets, no matter what happens. This 
means that we have to become more pro- 
ductive in one area, if another area exhib- 
its an “unavoidable” cost increase. 

In summary, we must find ways to design for 
lower costs, we must know our costs, and we 
must set out to meet those costs. . . 

In my years in the space program, and espe- 
cially during the years when I had some first- 
hand experience as a project manager, I 
gained a tremendous respect for American 
industry . . . That industry rose to the chal- 
lenge of Sputnik in 1957, and brought us to a 
position of world leadership in space. Today 
that same industry faces a new challenge — 
the challenge of doing more for less money. 

It is up to you to apply the same skills, inge- 
nuity and competitive spirit that allowed us 
to meet the challenge of 1957 to now meet 
the challenge of the 1970s — to preserve U.S. 
leadership in space through a productive pro- 
gram at a price the nation can afford. 
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